Virtual-environments: Ruby 2.6.3 not found - broken workflows without warning

[philipengberg]

Since last night, we're basically seeing this error output for all of our builds.

Run actions/setup-ruby@v1
##[error]Version 2.6.3 not found
##[error]Node run failed with exit code 1

[scelis]

Looks like they updated to 2.6.5. I would recommend changing your workflow to use:

      - name: Select Ruby
        uses: actions/setup-ruby@v1
        with:
          ruby-version: '2.6.x'

[zeisss]

We see this regularly as well, but every now and then things still work. Manually rerunning the builds also sometimes make them work.

[tisba]

I just realised that the README states that setup-ruby will not actually install requested Ruby versions. That's quite unfortunate :(

@scelis Your suggestion is behind the point. You want your CI run the exact version of Ruby that you are using in most cases.

I'd like to ask the maintainers/GitHub what the best practise is to get a specified version of Ruby installed and set up (regardless if its in the cache or not)...

[eregon]

https://github.com/eregon/use-ruby-action supports exact Ruby versions and all versions between 2.3.0 and 2.7.0.

Note that there are plans to install prebuilt versions for this action too in the future: https://github.com/actions/setup-ruby/issues/44

[jordoh]

https://github.com/eregon/use-ruby-action supports exact Ruby versions and all versions between 2.3.0 and 2.7.0.

Note that there are plans to install prebuilt versions for this action too in the future: actions/setup-ruby#44

Hi @eregon - while the 2-5 second action runs sound fast, that's not matching the from-cache speed this action offers:

Multiplied across a myriad of branches and repositories, the notion of adding time to every build is not ideal.

[eregon]

@jordoh Right, after all it must download the prebuilt Ruby from somewhere. The toolcache will never contain 30+ versions of Ruby so it will always be more limited.

I wonder though, what kind of workload do you run? Any bundle install or test run is going to take more than 5 seconds, and then 5 seconds is small compared to that.
In fact it's already a lot faster than on TravisCI or when using RVM.

[jordoh]

@eregon - it's not the test speed that I'm concerned with, but the overall time spent across all of the branches and repositories in my organization. It's not a huge concern at 2-5 seconds - my point is that switching from a ln to a download is a consideration for those of us with dozens of repositories, particularly when evaluating GitHub Actions against other paid solutions.

[rodrigopv]

https://github.com/eregon/use-ruby-action supports exact Ruby versions and all versions between 2.3.0 and 2.7.0.

We solved our today build issues changing setup-ruby for this action. It appears that now ruby 2.6.5 is being used sometimes by setup-ruby, and we have specified 2.6.3 on our Gemfile and .ruby-version.

[bryanmacfarlane]

Moving to virtual environments so they can comment on removing a version.

[al-cheb]

Hello, @philipengberg

Thank you for your time to report an issue. We have updated all Ruby versions on Windows, Ubuntu, MacOs images in the toolcache :

   Previous -> Current
2.4.6(replaced) -> 2.4.9
2.5.5(replaced) -> 2.5.7
2.6.3(replaced) -> 2.6.5
                   2.7.0(new)

Please check the documentation to find new updates:
MacOS GitHub 10.15 Week 3 (https://github.com/actions/virtual-environments/pull/283)
Ubuntu 18.04 - https://github.com/actions/virtual-environments/blob/releases/ubuntu18/20200113/images/linux/Ubuntu1804-README.md
Ubuntu 16.04 - https://github.com/actions/virtual-environments/blob/releases/ubuntu16/20200113/images/linux/Ubuntu1604-README.md
Windows Server 2016 - https://github.com/actions/virtual-environments/blob/releases/win16/20200113/images/win/Windows2016-Readme.md
Windows Server 2019 - https://github.com/actions/virtual-environments/blob/releases/win19/20200113/images/win/Windows2019-Readme.md

Example output:

    steps:
    - name: Use Ruby
      uses: actions/setup-ruby@v1
      with:
        ruby-version: '2.6'
    - name: Check Ruby version
      run: |
        ruby -v

[tisba]

@bryanmacfarlane Can you move the issue back?

I think the big issue here is that it is not clear that actions/setup-ruby does not install anything that is not already available. If you read the README closely you can probably guess it somehow. IMO there should be a very clear statement that actions/setup-ruby cannot setup/install arbitrary versions of Ruby.

[philipengberg]

@tisba Exactly, I agree.

[eregon]

Agreed, and it's already been reported as https://github.com/actions/setup-ruby/issues/46 but not addressed.

[eregon]

I made a PR to try to clarify this in the README: https://github.com/actions/setup-ruby/pull/52

[bryanmacfarlane]

This issue is tracking that when Ruby was updated (2.6.5 was added) 2.6.3 was removed with no warning and we had production escalations with folks that bound their workflows to 2.6.3 as that's what the readme said even as it was failing.

The readmes in this repo (and why the issue is in this repo) is not updated until the images roll to all machine pools which it does in a rolling multi-day safe rollout. That means if you follow the readme and bind to a version, your workflow will start failing one day with no notice and then days later, you will see a readme updated on what you should have updated your workflow to on an indeterminate date.

For that reason, the readme should encourage (and only support) binding to a minor version and setup-ruby can reference that. It's fine to list the exact versions but in the header, it needs to warn against using those versions.

[bryanmacfarlane]

It would also be nice if each tool (like Ruby) had a markdown ## header so other repos could index right to that tool with a link. e.g. setup-ruby readme.

[bryanmacfarlane]

I updated the guidance in setup-ruby in the meantime: https://github.com/actions/setup-ruby#setup-ruby

[MSP-Greg]

Ruby versioning major.minor.teeny

Some repos prefer to reference a particular Ruby version (m.m.t) in their CI, rather than a m.m version. In theory,

If one assumes that leaving all Ruby versions in the toolcache is not considered an option, maybe use a 'two version' system where anytime versions are updated, the most recent previous version is left in the toolcache?

Background:

Why do some people prefer to lock to a specific m.m.t version?

Within Ruby m.m releases, there should not be breaking changes.

But, items not part of 'core Ruby' maybe included in builds. One item in the past was Bundler, which until Ruby 2.6 was not included in Ruby, and was included at the discretion of the packager/builder (and if so, what version). Bundler is often used in CI to set up the test environment dependencies.

Additionally, the libraries that Ruby is built with are not really specified. An example would be Ruby 2.4, which was released before the release of OpenSSL 1.1.0, but is often now built with OpenSSL 1.1.1.

[bryanmacfarlane]

+1 to @MSP-Greg n - 1 suggestion.

[ahaverty]

I got caught with this issue this morning for 2.5.5.

Why do some people prefer to lock to a specific m.m.t version?

I'd be happy to wildcard the teeny version, but our team has multiple projects, with differing ruby versions. We use rbenv to manage switching, and find that specifying the version in .ruby-version and Gemfile for automatic switching works great.
I've given it a go with just m.m.x but rbenv seems to require at least m.m.t.
Anyone has a workaround for that? We'd be happy to not have to lock down .t if we could.

[MSP-Greg]

@ahaverty

At present, using Ruby in Actions via 'official' GitHub solutions/actions is somewhat in flux.

Some of this is due to the fact that the Ruby organization does not release binary builds, hence, the concept of 'official binary builds' doesn't exist, as it does in some other languages.

Also, in what way are EOL builds supported, and also how they're built. For example, right now, Ruby 2.4 is not quite EOL, but some builds of it use OpenSSL 1.0.2, which is technically EOL...

So, for the time being, consider https://github.com/ruby/setup-ruby. Several well known repos/gems are using it. Note that it's location may change in the future, but there should be 'fair warning' on that.

[alepauly]

(I'm with the team that builds and deploys the images from this repo.)

I think it was already mentioned somewhere above but our current policy is to roll the teeny (revision) upgrades with no notice under the assumption that most tools do not introduce breaking changes in those updates. Removal of major and minor versions will definitely carry some notice.

We've definitely haven't done a great job communicating that policy so far, I'll make sure that's documented better.

That said, there's no reason why we couldn't adjust for certain tools if they historically handle versioning differently. Is this the case for Ruby? We're open to come up with a policy that works best for you all that consume the images and is sustainable from a maintenance perspective.

As an aside (and something else to document better) with respect to the timing of rollouts and readme updates: as soon as new version of an image starts to roll out we publish the release branch with the updated readme file and add a release (marked as pre-release). When an image has reached all customers we update the readme in the master branch and mark the release as final.

And thanks for all the ideas and input here, I know we haven't been ideally responsive in this repo but we're working to fix that.

[MSP-Greg]

our current policy is to roll the teeny (revision) upgrades with no notice

That said, there's no reason why we couldn't adjust for certain tools if they historically handle versioning differently

As mentioned above, some people would prefer that their CI pins to a specific teeny release, allowing them to test before updating to a new release. From past experience with another CI provider, there was one time where the teeny updates also included an update to Bundler (which they included in their builds), and it broke a lot of CI.

So, would a two version rolling system work? For instance, I think the recent update went 2.6.3 -> 2.6.5. Could both be left in the toolcache, and when 2.6.7 is released, 2.6.3 would be removed? Hopefully, that would help the majority of people that want to fix their CI to exact versions...

As an aside, where are the scripts for your Ruby build code?

[alepauly]

@MSP-Greg

So, would a two version rolling system work?

I'll discuss this with our team to see what we can do in the short term, since it seems important for Ruby to support this, perhaps it's sustainable from an image perspective. I'll update here once we discuss it.

As an aside, where are the scripts for your Ruby build code?

Right now that code is in our private repo but we have plans to clean it up and make it public. We're thinking it will live at https://github.com/actions/virtual-environments-packages, but we'll know more once we can start working on it. I suspect that will be in the short to medium term.

[MSP-Greg]

@alepauly

I'll discuss this with our team to see what we can do in the short term

See https://github.com/actions/setup-ruby/pull/49. You're probably aware of it, speaking to having a complete set of Ruby binaries available for 'install on demand' when a version requested isn't in the toolcache. Only takes a few seconds to install.

Right now that code is in our private repo but we have plans to clean it up

Thanks, and understood.

[alepauly]

@MSP-Greg

See actions/setup-ruby#49. You're probably aware of it

Yes, @bryanmacfarlane has been keeping me informed but thanks for making sure. If we go with the n+n-1 it will at least reduce breakages until on-demand installs are available. We'll take that into account as we figure it out.

[thejoebourneidentity]

In the meantime, we should add 2.6.3 back to the image and go with the suggestion of

go with the n+n-1

[StingyJack]

As an aside (and something else to document better) with respect to the timing of rollouts and readme updates: as soon as new version of an image starts to roll out we publish the release branch with the updated readme file and add a release (marked as pre-release). When an image has reached all customers we update the readme in the master branch and mark the release as final.

@alepauly - (continuing from 1467 in the old repo) That schedule is not clear at all. Customers don't know what deployment group they are in, and there is only ever one published document we can look at for what the image contains.

Also this is not just about the version numbering in the third position, a few months ago nodejs was jumped from v10 to v12.

So to restate the problem as I understand it- we need to know about image changes that are going to affect us specifically before they are put into effect for us. "Before" means with enough time for us to address it. Two weeks seems reasonable to me, so long as its actually two weeks and not an old comcastic window of "sometime between 2-10 weeks depending on my deployment group/ring/whatever".

Putting the doc updates in a release branch and marking as pre-release doesnt help anyone because it means one more thing that we have to go pretty far out of the way to look for and manage for ourselves. The ideal notification for me would be something that tells me when I log into DevOps (as someone who has permission to author pipelines) and am using a specific version numbered component that is being replaced, that I get the notification about the replacement each day for two weeks ahead of my deployment group's scheduled rollout.

I think we recognize that as a service provider you are trying to provide the most useful service to us for the lowest cost for you, and that means these versions will shift over time. However, breaking these images on us _stops our businesses from working until we scramble to attend to the interruption and update our programs_. Consider the interrupt this can have in the middle of a busy sprint. To that end it would be nice would be the ability to lock, freeze, or specify the image we are using. You'd only have to hold or store that image until nobody was using it anymore, and it would give us customers that key thing we need from you as a service provider - Continuity of Service.

[MSP-Greg]

Having wrestled with moving several repos from Travis/AppVeyor to Actions, four thoughts:

1. GitHub has to draw a line somewhere in terms of what is pre-installed on their images.

2. If organizations are sensitive to version changes in their dependencies, they should develop solutions that are as CI independent as possible. From what I've seen, packaging them as release assets and downloading/extracting/installing them does take time, but it's not excessive.

3. GitHub could consider a rolling update system, where x versions of each tool are always available, but that still leaves a version being removed when image updates are done. I believe they're already doing so with many tools.

4. Look for solutions elsewhere. The particular problem that started this issue has been 'solved' by the Ruby community.

EDIT:
@StingyJack

Read the link you provided, and this issue (and what I addressed) is more about versions disappearing. You issue is more about changing 'default' versions...

[andrelaszlo]

2.6.5 just stopped working for us:

##[error]Version 2.6.5 not found

We had 2.6.x specified for the job, initially, but 2.6.5 set in the Gemfile. Was hoping that setting the minor version would solve this but this was not the case.

Edit: Using setup-ruby on @dentarg's recommendation.

[dentarg]

@andrelaszlo you might want to take a look at https://github.com/ruby/setup-ruby

[andrelaszlo]

@dentarg Tackar Patrik, we'll give it a try!

Edit: That worked well, thanks for the solution!

[maxim-lobanov]

Hello,
In current implementation actions/setup-ruby doesn't install ruby on-flight. It just switches versions that are pre-installed (pre-cached) on image.
On image, we keep only latest patch version for each x.y.* pair because we can't afford to put more versions on the image.
If you are using actions/setup-ruby it is recommended to specify only 2 numbers in version (like 2.4) and latest patch will be picked up. This way will work for the most of the pipelines because patch version doesn't bring breaking changes.
If your use-case requires freezing particular patch version, please consider using action that is provided by ruby community ruby/setup-ruby. This action installs versions on-flight and can download any version.

[Gauravbtc]

Hello,
I am trying to install Ruby 2.6.1 version in my workflow but it gave me below error

Is the workflow did not support Ruby 2.6.1??

[MSP-Greg]

@Gauravbtc

actions//setup-ruby only supports single major.minor versions of Ruby, normally the latest releases.

If you want older versions (like 2.6.1), use https://github.com/ruby/setup-ruby

[Gauravbtc]

@MSP-Greg
thanks for your suggestions but I used setup ruby workflow but even though it will rasing error please find below code
steps: - uses: actions/checkout@v1 - name: Setup Ruby uses: actions/setup-ruby@v1 with: ruby-version: 2.6.1

[MSP-Greg]

@Gauravbtc

No. You're using actions/setup-ruby. I suggested using ruby/setup-ruby.

If someone responds to a closed issue, please take the time to review what they've written.