Conda-forge.github.io: R 4.0.0 migration tasks

[beckermr]

This is a thread to track tasks for the R 4.0.0 migration.

• [x] enable opt-in automerge in the YAML migrations (https://github.com/regro/cf-scripts/pull/963)
• [x] add automerge to all feedstocks w/ the @conda-forge/r team and a CRAN url (https://github.com/conda-forge/admin-migrations/pull/27)
• [x] write the YAML for the migration (https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/552)
• [x] build r-base and r for 4.0.0
• [x] figure out CDN sync stuff (https://github.com/regro/cf-scripts/issues/961, https://github.com/regro/cf-scripts/pull/968)
• [x] chained circle jobs (https://github.com/regro/cf-scripts/issues/900, https://github.com/regro/circle_worker/pull/10)
• [x] figure out max_pin for the packages or something like this (@isuruf should fill in details here)
• [x] write mini-migrator to do cleanup of R recipes
• [x] test new R with gfortran and upstream R binaries
• [ ] do the thing

cc @conda-forge/core @conda-forge/r @isuruf @CJ-Wright @conda-forge/bot

[dpryan79]

It probably makes sense to starting trying to build some of the prereleases of R since otherwise it takes a week or two to get the thing to build :(

[beckermr]

Go for it @dpryan79!

[dpryan79]

I figured I was volunteering myself with that comment :P

[CJ-Wright]

@beckermr we don't need a custom class, we need a custom yaml, the bot won't be able to intuit the correct number of R pins (since we have two at the moment).

[beckermr]

What is a custom YAML?

[beckermr]

Do you simply mean we need to make the migration yaml ourselves?

[CJ-Wright]

This is the R release schedule (pulled from https://developer.r-project.org/)

Friday 2020-04-17: CODE FREEZE (4.0.0 RC)
Tuesday 2020-04-21: PRERELEASE
Friday 2020-04-24: RELEASE (4.0.0)

[CJ-Wright]

xref: https://github.com/conda-forge/r-base-feedstock/pull/123

[bgruening]

Updated link and checksum in https://github.com/conda-forge/r-base-feedstock/pull/123

Final release is out.

[CJ-Wright]

I think currently this is awaiting msys2 getting a pcre2 package.

[CJ-Wright]

Also power is stalling out

[isuruf]

I would also add cleaning up the meta.yaml as something to do. (license, windows compiler, etc.)

[beckermr]

figure out max_pin for the packages or something like this (@isuruf should fill in details here)

Can you explain this?

I would also add cleaning up the meta.yaml as something to do. (license, windows compiler, etc.)

Do you mean for the R recipes in general or just r-base?

[isuruf]

figure out max_pin for the packages or something like this (@isuruf should fill in details here)

We can ignore this and keep it as it is now.

Do you mean for the R recipes in general or just r-base?

For the R recipes in general.

[CJ-Wright]

You might take a look at the noarchr migrator. It might be good to turn that into a minimigrator.

[beckermr]

I think we need to decouple these maintenance tasks here. I’d rather not deal with a combination of errors in our scripts to update recipes and legit build errors when moving to R 4.0.

Let’s open issues for the R recipe cleanup and forge ahead with the migration.

[isuruf]

These would be linter failures, so you'd need to fix them manually because automerge will fail.

[beckermr]

These would be linter failures, so you'd need to fix them manually because automerge will fail.

Automerge ignores the linter

[beckermr]

see here: https://github.com/regro/cf-autotick-bot-action/blob/master/conda_forge_tick_action/automerge.py#L16

[isuruf]

Let me explain why noarch: generic R packages need to be built.

First of all, it's not noarch: runlike noarch: python. noarch: python does a few things,

1. Extract the files in the same format for OS+arch.
2. Create the correct site-packages directory and move the files there.
3. Create .pyc in post-linking stage
4. Other things like entry_points which is not really important for this discussion

What noarch: generic does is 1. only. For R packages 2 and 4 are not important.
Equivalent to 3 would be the .rdx files. Unlike python, R doesn't install regular .R files. Only these bytecode objects. So importing one from a different minor version gives a warning and a different major version gives an error.

This can be worked around by generating these at post-link stage for major versions (for minor versions, we can ignore the warning by changing the max_pin), but this needs support in conda.

TL;DR: noarch:generic is noarch for OS+arch only and needs a rebuild for each R version.

[beckermr]

Thank you for finally posting this info per the request at the top of the comments! I am going to have a pass at adding a rebuild noarch option. Hopefully, this doesn't confuse the bot terribly. We are stopped right now because another bug appears to not have been fixed.

[CJ-Wright]

The migration now include noarch packages

[CJ-Wright]

Great work everyone!!

[jakirkham]

Wow! Does that mean we are done completely?! If so, this has to be some kind of record 😄

[beckermr]

Everything but a few tens of packages that depend on r-rjava and a handful of stragglers as usual. ;p

[CJ-Wright]

The automerge combined with the bot is truly spectacular. I'm looking forward to seeing how this impacts the python 3.9 rollout.

[beckermr]

And mamba. It doesn’t work without mamba.

[CJ-Wright]

Truth

[jakirkham]

Everything but a few tens of packages that depend on r-rjava and a handful of stragglers as usual. ;p

That sounds done to me 😉

Great work! 😄