Julia: Checklist of steps towards 0.5 release [candidates]

Created on 14 Jul 2016  ·  84Comments  ·  Source: JuliaLang/julia

[minor edits / cross referencing welcome here, but copy-paste your changes before submitting in case of multiple people trying to simultaneously make edits and be careful not to overwrite changes made by others - for additions, or deletions?, leave comments below and we'll discuss before adding to the master list]

Timeline for things that need to happen in a specific order:

  • [x] finish the remaining breaking/feature items on the 0.5.0 milestone

    1. SSH support in Pkg: https://github.com/JuliaLang/julia/issues/16041, PR at https://github.com/JuliaLang/julia/pull/17391 - assignees = Art, Keno, and myself

    2. safe non-traditional array indexing: https://github.com/JuliaLang/julia/issues/16973, (possibly stale?) PR at https://github.com/JuliaLang/julia/pull/17228 - assignees = Tim, Jeff, core team for feedback

  • [x] announce feature freeze by changing VERSION from 0.5.0-dev to 0.5.0-pre #17503

    • master becomes (temporarily) locked for anything other than bugfixes (preferably release-critical ones, anything not absolutely critical can potentially wait until backporting for 0.5.1), doc fixes or test additions

  • [x] create release-0.5 branch, change master VERSION from 0.5.0-pre to 0.6.0-dev to reopen master for breaking changes

    • this could happen immediately after the feature freeze step, but unless progress towards breaking 0.6 changes is inconveniently blocked here we might want to wait a short time

  • [x] at this point commits should not be made directly to the release-0.5 branch without prior discussion, the branch will operate in release mode:

    1. bugfixes should be made as usual via PR's to master first, wherever applicable there

    2. if you think something is a backport candidate, ask about it with a comment, we will have some discussion and decide yes or no on backporting during the RC process

    3. a contributor will mark the issue/PR with the backport pending 0.5 label, or ping the @juliabackports user in a comment (which sends a message to https://groups.google.com/forum/#!forum/julia-backports which we will monitor) if there's a needed commit to master without an associated issue/PR number (we should almost always avoid doing that)

    4. backporting changes should be done as follows:



      • git checkout release-0.5


      • git pull


      • git checkout -b $USER/backport-foo


      • git cherry-pick -x -e SHA


      • Edit the commit message to add "ref #..." so the cross-reference link shows up to the associated issue/PR.


      • Open a pull request targeted against the release-0.5 branch (there's a github dropdown menu on the open PR page) so CI runs. Please add [release-0.5] to the title of the PR so it's more obvious that the PR is against the release branch and not master.


      • If we have a lot of these, I will handle the backporting myself with a series of rollup pull requests to lessen the CI load.



  • [x] when the last RC1-critical cleanup items (see below) are finished, we tag v0.5.0-rc1, build binaries, and announce to the mailing lists

    • automatic windows code signing is not working right now on the buildbots, so this will have to be done manually on a windows machine by someone who has access to the certificates - probably me

    • any users and package authors who've been waiting to test their code and update their packages for 0.5 should do so

    • the most recent RC will be available under Travis language: julia as a julia: 0.5 option and AppVeyor with an 0.5-latest url, but release will not change until 0.5.0 final is tagged

  • [x] repeat the last few steps in the process, targeting 1 RC per week, until we're satisfied with being ready to tag v0.5.0 final.

    • I'll probably handle most of the backports at this stage, and run PackageEvaluator on the proposed changes to make sure we don't introduce regressions between successive RC's

  • [x] tag v0.5.0 final, celebrate, :fireworks: :beers: etc, get 0.6 going and onwards towards 1.0, world domination

    • bugfix backport 0.5.x releases will happen at a target of one per month, as we've been doing

    • change VERSION on release-0.5 branch to 0.5.1-pre

Things that can happen at any time but need to be done (help welcome!) before RC1:

  • [x] remaining NEWS and doc additions for Stringapalooza, threads, IOContext https://github.com/JuliaLang/julia/issues/16763 and all other breaking changes, deprecations, and major feature additions and performance changes that users should know about (e.g. startup time on Windows, LLVM version, debugger availability, iterator traits #15977, #17439, etc)
  • [x] deprecations still not in NEWS like #16603 and many others (done in #17779)
  • [x] make sure CI, buildbots, and tests in both source builds and binary installs are all happy with the final set of changes (linux binaries need https://github.com/JuliaLang/julia/issues/17602#issuecomment-237190583 fixed prior to 0.5.0 final to get rid of a warning when libgfortran is not installed system-wide)
  • [x] make sure make release-candidate passes and checklist will all work, and keep it that way (doctests etc)
  • [x] fix final important straggler bugs, e.g. Pkg.checkout issues, deciding what to do about ARM/Power breakage (maybe use a different LLVM version but only on those architectures, if that's the best fix)

Things that can start now but will go on between RC1 and v0.5.0 final:

  • [ ] fixing broken packages! see http://pkg.julialang.org/pulse.html, there's a lot of work to do to bring the ecosystem to a state where it works as well on 0.5 as it does on 0.4 right now
  • [x] identifying and fixing any release-blocking bugs that we don't know about yet and/or can't fix before we decide to tag RC1
  • [x] review parts of the documentation that aren't tested by doctests and make sure they're up to date (moved from rc1 to ongoing)
  • [x] review issues and PR's (both open and closed) marked with needs-docs and needs-tests and rectify any we can (moved from rc1 to ongoing)

The later pieces of this have been my policy for release management ever since the 0.3.x backports and 0.4 RC's, which has been working pretty well from my impression. Feedback on this is welcome, and I'll be turning these items into a formal written release/backporting policy document going forward.

Most helpful comment

will be tagging, building and uploading today

All 84 comments

I would like to see #17422 added to the 0.5.0 milestone.

I'd like to see #17423 added to the 0.5.0 milestone.

(I'm adding all of these as separate comments so that people can like, dislike individually).

I would like to see https://github.com/eschnett/SIMD.jl/issues/12 added to the 0.5.0 milestone. Although reported against the SIMD package, this seems to be an issue with setindex! for arrays in Base.

See the final line of https://github.com/JuliaLang/julia/pull/16125#issuecomment-216537524 (I'm not sure what the right thing to do is, but I suspect we should do something).

@timholy are you responding to someone or making a separate point? I think something could be added to PkgDev for ambiguity checking. Or PkgEval, or the default Travis script.

@eschnett looks like you closed that in your package. just in terms of how github works, something needs to have an issue or pr in this repo to be added to the milestone.

I'd like to see #17438 added to 0.5.0.

I meant that if it needs to be a change to Pkg itself (e.g., add another keyword to test, with the default behavior...to warn?...to error?...to not even check?), we should do that before RC because it could have _substantial_ impact on people who are busily updating their packages.

If the change will be applied elsewhere, then no sweat.

Doing it in Pkg.test would get more people running it right away, so there would be a big benefit to doing it there. I don't think anyone was planning on working on anything related to ambiguity, but it would be a good idea. Maybe warn by default with a kwarg to disable (or set to an error) would be good?

detect_ambiguities needs to be passed a module name. What's the right strategy? I'm guessing it's rather error-prone to assume there is a module with the same name as the package?

Or make the keyword be a list of modules that the user wants to test? It will thus require opt-in (which has its strengths and weaknesses).

We should move this to its own issue or placeholder for a PR (edit: done by Tim, https://github.com/JuliaLang/julia/issues/17444). Doesn't the package loading machinery already assume the module and package name are the same?

@tkelman We found and corrected the bug (which was in SIMD after all).

I'd like to see #17386 issue fixed in 0.5.0

Also #15017, #14495 and #16746 issues...

@tmptrash: best reference issues & PRs with just typing: #17386. That way one sees the number and hovering the mouse-pointer over the link will show the issue title. Thus your issues are: #17386, #15017, #14495, and #16746.

I'd like to see #17481 fixed in 0.5.0.

@tmptrash your issues should be addressed but I don't see them as release blocking. They all seem to be featured around Tasks which, while it's important to fix, it's not like people can't really run Julia v0.5 without them fixed.

The remaining issues should focus on things like making sure the new package manager works correctly on Windows (@davidanthoff 's issues) or making sure that the Rmath.jl change works (as of right now it breaks all of the stats libraries, some plotting packages, some optimization packages, etc.). These issues are hard for (at least Windows users) to NOT run into.

I agree, that some of them are not blockers. Should i remove them from the list? On the other hand #15017 is a critical, because it produces crashes. I have waiting for this fix since february and as i understand, it has already fixed in a brunch. Anyway i can agree with your desicion ;) In this #17386 issue, the part of language (profiler) doesn't work. So maybe it also a little bit critical :)

P.S. I'm new in julia and not so deep understand all the process, sorry.

Just because it produces crashes doesn't make it critical. #15017 produces crashes if you use eval in a loop with Tasks. That's a pretty specific case which I have not seen in any package. In fact, it's been an issue for quite awhile, and the vast majority of Julia simply avoided doing it. So I don't think that large parts of Julia v0.5 are unusable without fixing it, and that seems like a good working definition of what's release blocking.

Since there's already a fixed branch for it, it'll get tested and probably put into a master build soon. However, I don't see a major release being delayed for it.

Okay, what about #17386?

Same thing. You are clearly pushing into new territory with Tasks, which is great, but it doesn't affect 99.9% of all users of the profiler.

That again looks like the same issue: Tasks with eval in a loop. Can you come up with a way of creating this error without using Tasks with eval in a loop? If not, then it's not really critical because unless you have this exact use case (which, try to find any Julia package which does this) you don't get this error.

I think it's because eval is not thread safe, yet you're using it with a parallelization construct that requires some kind of safety (though I am not really sure). It sounds dangerous to mix the two. Yes, a good fix for it should definitely come at least by the time threading is introduced as non-experimental (which I think is by v1.0?), but it's not part of what Julia v0.5 is really trying to do.

I don't need real threads. The idea of green threads is a best solution for my task, because in reality there is only one system thread and i don't need to synchronize data between green threads. This is one of reasons, why i chose julia for my research. Anyway, i'm waiting for v0.4.7, where this issue should be fixed.

if it's already been fixed on master, then it certainly doesn't need to be mentioned here

If the bump to 0.6.0-dev is not the first commit after a tag, the "build number" won't be reset. We might want to fix that first (or along with the version bump, at least). See https://github.com/JuliaLang/julia/pull/17434#issuecomment-233862489 for an idea.

17107 should be added to the 0.5 milestone. It's only an issue with an example in the manual but it should be addressed before release.

I am of the opinion that the regressions in https://github.com/JuliaLang/julia/issues/16128#issuecomment-226636627 should at be looked into more than has been done yet before tagging a 5.0 release. This is of course not RC blocking and it might not be possible to fix all regressions but for a language where performance is of big importance it makes sense to make sure that a new release does not introduce known regressions without an explanation as to why they occur and a plan to fix them in the future.

@JeffBezanson, @vtjnash, @yuyichao, @Keno and others have looked into all of these regressions and done a huge amount of work to address them. There are a few known performance issues remaining, but most have been addressed. Further work on performance will continue in 0.5.x releases, but we need to get a 0.5.0 release out ASAP. If something isn't RC blocking then it's not release blocking (by definition).

Ok, it seemed that most of the work was done towards the "macro benchmarks" in the first post in that issue and less on the micro benchmarks that I listed in my comment but maybe things have changed (improved) since that comparison was run. Would be interesting to see a new full v0.4 vs master nanosoldier run. Anyway, I understand what you are saying so lets work on it for 0.5.1 then :)

13472? An release (candidate) is likely to be used by a larger audience and see more users behind firewalls.

I think #17560 needs to be release blocking and not be assigned to 0.5.x. Secure handling of user passwords should not be left to point releases.

As of #17585, NEWS now addresses the Stringapalooza changes, I believe.

I'm going to propose that we tag 0.5.0-rc0 as a very complete beta release. We still know there are a couple minor issues (doc tests, miscellaneous minor bug fixes), but tagging rc0 will signal a) that this is actually happening _now_ and b) that people should start using it, which we really need.

Yeah, let's tag rc0 and get the binaries out in people's hands to start testing. Will also start getting packages moving towards 0.5.

Okay, we should be able to do that. Since most of the remaining bug/doc fixes will also be needed for 0.6 anyway and there don't seem to be any urgent blocked 0.6 issues yet, we can hold off on branching until we're closer and ready for an rc1 that we're happier with.

@vtjnash, as far as https://github.com/JuliaLang/julia/issues/16091#issuecomment-234824825

Adding this to the milestone since it needs a breaking change to the serializer protocol.

is concerned, what visible consequence will that have if people start testing on an rc0 that is made prior to that breaking change, and rc1 and later are made after? Anything more noticeable than serializer output saved with pre-16091-fix-julia can't be loaded by post-16091-fix-julia?

I think there can be an rc1 (or rc0 if you must) as soon as we know we're done with breaking changes and deprecations, such that it makes sense to start updating packages.

that's what feature freezing -pre was for...

... and it somehow relates to the 'nowish' deadline communicated at JuliaCon ...

I think we should do rc0 right away - with wide announcement for package authors. With rc1 we should request end users to start testing and updating as well.

This baby is definitely read for consumption by package authors so that the package updating can start, from my point of view. Not ready for a release, because of a number of bugs/issues, but @StefanKarpinski suggestions of an RC0/beta sounds right to me.

Will tag rc0 after merging https://github.com/JuliaLang/julia/pull/17634, since I accidentally broke the from-scratch build last night. I don't think I'm going to do the manual code signing on the windows binaries for this one if we know it isn't final.

I'd like to see #17364 added to the list of releasing blocking regressions. That bug messes up local package repos on a _very_ common code path, so that even if a 0.5.1 release fixed the bug, people would have to manually clean up the branch mess that #17364 creates before things would work again.

@davidanthoff It's annoying, but it's not a showstopper. And there are other Pkg bugs. And other usability issues.

Did we ever announce the rc0 binaries anywhere yet, and what we want people to do with them? Also what else holds up rc1 now? The 0.5.0 milestone tag doesn't have anything on it except this issue and a NEWS update.

we should try to fix a few more of the pkg bugs if we can. there are open pr's for at least 1 but they need a bit of test tweaking. doctests are also failing, I need to have a look at fixing the rest of those. we did not announce rc0 yet as far as I know, I thought Stefan was going to do so but don't know what else he was waiting for if anything. let's target Monday or Tuesday for rc1 and branching.

RC0 happened? There was certainly no announcement and even when watching things fairly closely it was easy to miss (I did).

@lobingera if #17364 really isn't added as a release blocker, someone should at least put an error() call at the top of Pkg.checkout. That would at least prevent data corruption of repos. But really, if that bug is not a show stopper, I don't really know what would...

@davidanthoff As RC0 showed up here https://github.com/JuliaLang/julia/releases, it wasn't really hard to recognize it...
I agree with you #17364 could create repo problems and keep some people from working (i even think, i would be affected). Still, you could operate a development with command line git.
And btw: if you already know how to update Pkg.checkout code to avoid problems, it might be an idea you adress the issue, provide testing and run the PR. If there is a working solution it still might make it into 0.5

RC0 is tagged, but not announced. I think this release is aimed for students who are currently working on GSoC. They are those who need it the most.

It is aimed for everyone (not just the GSOC students) and we are waiting for @tkelman to release the binaries.

no you aren't, binaries have been up since the day after the tag.

@tkelman Can you post the URLs of the binaries here?

Announced on julia-users:

https://groups.google.com/forum/#!topic/julia-users/shrnNHpyliA

https://github.com/JuliaLang/julia/issues/17364 has the advantage of an open PR at #17399, we know what the fix is but need to make sure the test for it is representative (i.e. the test would fail without the fix) so it doesn't regress. I think https://github.com/JuliaLang/julia/issues/17610 actually has more serious consequences in regular use since it prevents packages from being updated or removed in normal use, but we haven't fully debugged that one or come up with a fix yet.

I think doing RC's on a time-based schedule for the next couple of weeks, one per week, makes the most sense. We'll get as many bugfixes and cleanups in as we can, and make a judgement call when we get ready for the next RC whether or not it's ready to be called 0.5.0 final. RC1 definitely is not, but if we tag it on Monday or Tuesday then it'll have a few more bugs fixed than RC0.

Apologies for being late to the party, but I'd like to see #17109/#17110 addressed in 0.5 (they are regressions from 0.4).

@jballanc: those are not going to be show stoppers for the 0.5 release but I've tagged them 0.5.x.

Yeah, I don't know whether we'll have time to fix those for 0.5.0 but they should have at least gotten triaged properly, probably got lost during JuliaCon.

@ViralBShah Could you indicate in your announcement emails to the larger user base whether a given RC is actually a candidate for a release or not? For example, for RC1, just add a sentence a la "We tagged RC1. RC1 is not a candidate for release because it has known bugs and there will be at least a RC2" or something like that. And then when there is an RC that you guys actually think is done, obviously also indicate that in the email. That sort of information really, really helps us people downstream.

The main reason to mention the URL to this page was to signal exactly that "this is mostly ready, but there are still a couple small issues - but we want you guys to give it a try".

The email could certainly be clearer about this.

fixing broken packages! see http://pkg.julialang.org/pulse.html, there's a lot of work to do to bring the ecosystem to a state where it works as well on 0.5 as it does on 0.4 right now

Is there a separate discussion around this, possibly for divvying up work for those who would be willing to update others packages?

Also, when I look at http://pkg.julialang.org/ there are a lot of packages that had their last 0.5 test result from a couple of months ago. Might make sense to run these package tests for all packages on RC0, so that the pass/fail results reflect how packages fare on the RC0 build, and not a many months old 0.5 build?

I think the testing status is up to date, but it's only testing the latest tagged release, which is what the date is.

@Keno Ah, you are right, I misinterpreted the labels there.

Was hoping we could get https://github.com/JuliaLang/julia/issues/17364 resolved over the weekend, but that moved a little slowly. I think if https://github.com/JuliaLang/julia/pull/17773, https://github.com/JuliaLang/julia/pull/17777, and https://github.com/JuliaLang/julia/pull/17779 get reviewed and look ready to merge then we'll branch and tag rc1 today.

We have branched. https://github.com/JuliaLang/julia/pull/17389 was a big performance regression, not entirely fixed by https://github.com/JuliaLang/julia/pull/17798, so I'm going to propose reverting it on the release-0.5 branch and making rc1 from that right now.

0.5.0-rc1 was tagged, with one more reversion because #17522 is causing test failures on Windows. Should have branched from before that was merged, but if the test failures can be resolved then we can consider reverting the revert and backporting the test fix for rc2.

Binaries are building now on the buildbots and will be ready soon. I'll refresh my memory on how to do the Windows code signing this time, then announce on the mailing list once everything is moved around to the julialang bucket on S3. The links will be the same, just replace rc0 with rc1.

Is there a separate discussion around this, possibly for divvying up work for those who would be willing to update others packages?

Not yet, and I'm not sure whether this repo is the best place to do so. Mailing list? Gitter? Placeholder repo in some organization?

A decent number of packages may be working okay on master but need tags, so in those cases some prodding to release a new version that passes on 0.5 may be in order. It wouldn't be too hard to modify PkgEval to test against master of everything if someone's interested in trying that and seeing how the results differ.

@randyzwitch as Tony Kelman said, for most largely used packages (PyPlot, Juno, all of JuliaStats, etc.) you'll probably notice that people already have it working on v0.5-rc0 but you just need to checkout master (likely on many dependencies as well). It would be best to work it out via Gitter or however else you can get in contact with the main developers for this reason. For example, to get Juno working on v0.5-rc0, you just have to checkout master on like 8-9 different repos. So the issue is mostly about tagging the new versions.

The only major issue would be to get everyone to root out the deprecations. If you use something like PyPlot, you get a wall of deprecation warnings, usually the string deprecations. It works but is off-putting. Someone should use PkgEval test and go around to fix the deprecations in the large projects, or at least the ones that many other packages depend on.

Just want to make it very clear that I echo @davidanthoff 's request above.
https://github.com/JuliaLang/julia/issues/17418#issuecomment-236679744

As an update here, I'm preparing a branch full of cherry-picks of backport pending PR's for RC2, which I'll be opening a PR for when it's ready to run nanosoldier and pkgeval on. (It will look a lot like master, minus a handful of things that aren't backport candidates.)

edit: #17960

rc2 will be tagged momentarily and uploaded as soon as it's built has been tagged and uploaded, just need to announce

Is rc2 actually literally a release candidate, unlike rc0 which was understood not to be, but more like a beta release?

rc2 is missing one major thing at the moment, which is proxy support - that exists on master but the build system with curl is rough around the edges and will need some work to get backported into rc3.

I'd like to nominate #17146 (or #17995, they might be dups) for 0.5.0. They corrupt repo state and I don't think 0.5.0 should ship with bugs on such a common code path that messes up local repositories.

RC3 backports are at #18156. This includes curl and should work with proxies, so will be the first release candidate that we could seriously consider promoting to 0.5.0 final. We'll decide after between a few days and a week of testing on RC3 whether to put out an RC4 or go straight to final.

See #18276 for RC4 backports. It doesn't look like many more bugfixes are going to happen right now, so anything that doesn't have a fix materialize soon will be 0.5.1 material.

I am wondering, should the issue described in #18353 get a NEWS mention (in short: non-obvious behaviour when passing redefined functions to map, quadgk etc. due to #265)?

I ran into this the moment I used 0.5 for interactive numeric work, and it took a moment to figure out why things were behaving strangely, since it works without problems on 0.4.

There doesn't seem to be a step in the release process for adding release-0.5 to readthedocs.org.

At the moment on docs.julialang.org you have versions release-0.3, release-0.4, stable and latest. For me it's not clear which one is currently for v0.5. Thank you.

Yes, good question @Ken-B. I have never touched readthedocs myself, @StefanKarpinski who has admin access there to create a new release option?

I now have admin access and enabled the release-0.5 branch and the 0.5 docs are live: http://docs.julialang.org/en/release-0.5/

What needs to be done for the stable branch on readthedocs?

Is there going to be a RC5 for this stuff?

I'll post links to the binaries that are going to run through pkgeval, but the diff there is pretty small inside src and base.

will be tagging, building and uploading today

Was this page helpful?
0 / 5 - 0 ratings

Related issues

iamed2 picture iamed2  ·  3Comments

wilburtownsend picture wilburtownsend  ·  3Comments

yurivish picture yurivish  ·  3Comments

ararslan picture ararslan  ·  3Comments

StefanKarpinski picture StefanKarpinski  ·  3Comments