[minor edits / cross referencing welcome here, but copy-paste your changes before submitting in case of multiple people trying to simultaneously make edits and be careful not to overwrite changes made by others - for additions, or deletions?, leave comments below and we'll discuss before adding to the master list]
Timeline for things that need to happen in a specific order:
VERSION
from 0.5.0-dev
to 0.5.0-pre
#17503release-0.5
branch, change master VERSION
from 0.5.0-pre
to 0.6.0-dev
to reopen master for breaking changesrelease-0.5
branch without prior discussion, the branch will operate in release mode:backport pending 0.5
label, or ping the @juliabackports
user in a comment (which sends a message to https://groups.google.com/forum/#!forum/julia-backports which we will monitor) if there's a needed commit to master without an associated issue/PR number (we should almost always avoid doing that)git checkout release-0.5
git pull
git checkout -b $USER/backport-foo
git cherry-pick -x -e SHA
release-0.5
branch (there's a github dropdown menu on the open PR page) so CI runs. Please add [release-0.5]
to the title of the PR so it's more obvious that the PR is against the release branch and not master.language: julia
as a julia: 0.5
option and AppVeyor with an 0.5-latest
url, but release
will not change until 0.5.0 final is taggedVERSION
on release-0.5
branch to 0.5.1-pre
Things that can happen at any time but need to be done (help welcome!) before RC1:
make release-candidate
passes and checklist will all work, and keep it that way (doctests etc)Things that can start now but will go on between RC1 and v0.5.0 final:
The later pieces of this have been my policy for release management ever since the 0.3.x backports and 0.4 RC's, which has been working pretty well from my impression. Feedback on this is welcome, and I'll be turning these items into a formal written release/backporting policy document going forward.
I would like to see #17422 added to the 0.5.0 milestone.
I'd like to see #17423 added to the 0.5.0 milestone.
(I'm adding all of these as separate comments so that people can like, dislike individually).
I would like to see https://github.com/eschnett/SIMD.jl/issues/12 added to the 0.5.0 milestone. Although reported against the SIMD package, this seems to be an issue with setindex!
for arrays in Base.
See the final line of https://github.com/JuliaLang/julia/pull/16125#issuecomment-216537524 (I'm not sure what the right thing to do is, but I suspect we should do something).
@timholy are you responding to someone or making a separate point? I think something could be added to PkgDev for ambiguity checking. Or PkgEval, or the default Travis script.
@eschnett looks like you closed that in your package. just in terms of how github works, something needs to have an issue or pr in this repo to be added to the milestone.
I'd like to see #17438 added to 0.5.0.
I meant that if it needs to be a change to Pkg
itself (e.g., add another keyword to test
, with the default behavior...to warn?...to error?...to not even check?), we should do that before RC because it could have _substantial_ impact on people who are busily updating their packages.
If the change will be applied elsewhere, then no sweat.
Doing it in Pkg.test would get more people running it right away, so there would be a big benefit to doing it there. I don't think anyone was planning on working on anything related to ambiguity, but it would be a good idea. Maybe warn by default with a kwarg to disable (or set to an error) would be good?
detect_ambiguities
needs to be passed a module name. What's the right strategy? I'm guessing it's rather error-prone to assume there is a module with the same name as the package?
Or make the keyword be a list of modules that the user wants to test? It will thus require opt-in (which has its strengths and weaknesses).
We should move this to its own issue or placeholder for a PR (edit: done by Tim, https://github.com/JuliaLang/julia/issues/17444). Doesn't the package loading machinery already assume the module and package name are the same?
@tkelman We found and corrected the bug (which was in SIMD after all).
I'd like to see #17386 issue fixed in 0.5.0
@tmptrash: best reference issues & PRs with just typing: #17386
. That way one sees the number and hovering the mouse-pointer over the link will show the issue title. Thus your issues are: #17386, #15017, #14495, and #16746.
I'd like to see #17481 fixed in 0.5.0.
@tmptrash your issues should be addressed but I don't see them as release blocking. They all seem to be featured around Tasks which, while it's important to fix, it's not like people can't really run Julia v0.5 without them fixed.
The remaining issues should focus on things like making sure the new package manager works correctly on Windows (@davidanthoff 's issues) or making sure that the Rmath.jl change works (as of right now it breaks all of the stats libraries, some plotting packages, some optimization packages, etc.). These issues are hard for (at least Windows users) to NOT run into.
I agree, that some of them are not blockers. Should i remove them from the list? On the other hand #15017 is a critical, because it produces crashes. I have waiting for this fix since february and as i understand, it has already fixed in a brunch. Anyway i can agree with your desicion ;) In this #17386 issue, the part of language (profiler) doesn't work. So maybe it also a little bit critical :)
P.S. I'm new in julia and not so deep understand all the process, sorry.
Just because it produces crashes doesn't make it critical. #15017 produces crashes if you use eval in a loop with Tasks. That's a pretty specific case which I have not seen in any package. In fact, it's been an issue for quite awhile, and the vast majority of Julia simply avoided doing it. So I don't think that large parts of Julia v0.5 are unusable without fixing it, and that seems like a good working definition of what's release blocking.
Since there's already a fixed branch for it, it'll get tested and probably put into a master build soon. However, I don't see a major release being delayed for it.
Okay, what about #17386?
Same thing. You are clearly pushing into new territory with Tasks, which is great, but it doesn't affect 99.9% of all users of the profiler.
That again looks like the same issue: Tasks with eval in a loop. Can you come up with a way of creating this error without using Tasks with eval in a loop? If not, then it's not really critical because unless you have this exact use case (which, try to find any Julia package which does this) you don't get this error.
I think it's because eval is not thread safe, yet you're using it with a parallelization construct that requires some kind of safety (though I am not really sure). It sounds dangerous to mix the two. Yes, a good fix for it should definitely come at least by the time threading is introduced as non-experimental (which I think is by v1.0?), but it's not part of what Julia v0.5 is really trying to do.
I don't need real threads. The idea of green threads is a best solution for my task, because in reality there is only one system thread and i don't need to synchronize data between green threads. This is one of reasons, why i chose julia for my research. Anyway, i'm waiting for v0.4.7, where this issue should be fixed.
if it's already been fixed on master, then it certainly doesn't need to be mentioned here
If the bump to 0.6.0-dev
is not the first commit after a tag, the "build number" won't be reset. We might want to fix that first (or along with the version bump, at least). See https://github.com/JuliaLang/julia/pull/17434#issuecomment-233862489 for an idea.
I am of the opinion that the regressions in https://github.com/JuliaLang/julia/issues/16128#issuecomment-226636627 should at be looked into more than has been done yet before tagging a 5.0 release. This is of course not RC blocking and it might not be possible to fix all regressions but for a language where performance is of big importance it makes sense to make sure that a new release does not introduce known regressions without an explanation as to why they occur and a plan to fix them in the future.
@JeffBezanson, @vtjnash, @yuyichao, @Keno and others have looked into all of these regressions and done a huge amount of work to address them. There are a few known performance issues remaining, but most have been addressed. Further work on performance will continue in 0.5.x releases, but we need to get a 0.5.0 release out ASAP. If something isn't RC blocking then it's not release blocking (by definition).
Ok, it seemed that most of the work was done towards the "macro benchmarks" in the first post in that issue and less on the micro benchmarks that I listed in my comment but maybe things have changed (improved) since that comparison was run. Would be interesting to see a new full v0.4 vs master nanosoldier run. Anyway, I understand what you are saying so lets work on it for 0.5.1 then :)
I think #17560 needs to be release blocking and not be assigned to 0.5.x. Secure handling of user passwords should not be left to point releases.
As of #17585, NEWS now addresses the Stringapalooza changes, I believe.
I'm going to propose that we tag 0.5.0-rc0 as a very complete beta release. We still know there are a couple minor issues (doc tests, miscellaneous minor bug fixes), but tagging rc0 will signal a) that this is actually happening _now_ and b) that people should start using it, which we really need.
Yeah, let's tag rc0 and get the binaries out in people's hands to start testing. Will also start getting packages moving towards 0.5.
Okay, we should be able to do that. Since most of the remaining bug/doc fixes will also be needed for 0.6 anyway and there don't seem to be any urgent blocked 0.6 issues yet, we can hold off on branching until we're closer and ready for an rc1 that we're happier with.
@vtjnash, as far as https://github.com/JuliaLang/julia/issues/16091#issuecomment-234824825
Adding this to the milestone since it needs a breaking change to the serializer protocol.
is concerned, what visible consequence will that have if people start testing on an rc0 that is made prior to that breaking change, and rc1 and later are made after? Anything more noticeable than serializer output saved with pre-16091-fix-julia can't be loaded by post-16091-fix-julia?
I think there can be an rc1 (or rc0 if you must) as soon as we know we're done with breaking changes and deprecations, such that it makes sense to start updating packages.
that's what feature freezing -pre was for...
... and it somehow relates to the 'nowish' deadline communicated at JuliaCon ...
I think we should do rc0 right away - with wide announcement for package authors. With rc1 we should request end users to start testing and updating as well.
This baby is definitely read for consumption by package authors so that the package updating can start, from my point of view. Not ready for a release, because of a number of bugs/issues, but @StefanKarpinski suggestions of an RC0/beta sounds right to me.
Will tag rc0 after merging https://github.com/JuliaLang/julia/pull/17634, since I accidentally broke the from-scratch build last night. I don't think I'm going to do the manual code signing on the windows binaries for this one if we know it isn't final.
I'd like to see #17364 added to the list of releasing blocking regressions. That bug messes up local package repos on a _very_ common code path, so that even if a 0.5.1 release fixed the bug, people would have to manually clean up the branch mess that #17364 creates before things would work again.
@davidanthoff It's annoying, but it's not a showstopper. And there are other Pkg bugs. And other usability issues.
Did we ever announce the rc0 binaries anywhere yet, and what we want people to do with them? Also what else holds up rc1 now? The 0.5.0 milestone tag doesn't have anything on it except this issue and a NEWS update.
we should try to fix a few more of the pkg bugs if we can. there are open pr's for at least 1 but they need a bit of test tweaking. doctests are also failing, I need to have a look at fixing the rest of those. we did not announce rc0 yet as far as I know, I thought Stefan was going to do so but don't know what else he was waiting for if anything. let's target Monday or Tuesday for rc1 and branching.
RC0 happened? There was certainly no announcement and even when watching things fairly closely it was easy to miss (I did).
@lobingera if #17364 really isn't added as a release blocker, someone should at least put an error() call at the top of Pkg.checkout. That would at least prevent data corruption of repos. But really, if that bug is not a show stopper, I don't really know what would...
@davidanthoff As RC0 showed up here https://github.com/JuliaLang/julia/releases, it wasn't really hard to recognize it...
I agree with you #17364 could create repo problems and keep some people from working (i even think, i would be affected). Still, you could operate a development with command line git.
And btw: if you already know how to update Pkg.checkout code to avoid problems, it might be an idea you adress the issue, provide testing and run the PR. If there is a working solution it still might make it into 0.5
RC0 is tagged, but not announced. I think this release is aimed for students who are currently working on GSoC. They are those who need it the most.
It is aimed for everyone (not just the GSOC students) and we are waiting for @tkelman to release the binaries.
no you aren't, binaries have been up since the day after the tag.
@tkelman Can you post the URLs of the binaries here?
Take the existing binary links and just replace the version number, same as always.
https://s3.amazonaws.com/julialang/bin/linux/arm/0.5/julia-0.5.0-rc0-linux-arm.tar.gz
https://s3.amazonaws.com/julialang/bin/linux/x64/0.5/julia-0.5.0-rc0-linux-x86_64.tar.gz
https://s3.amazonaws.com/julialang/bin/linux/x86/0.5/julia-0.5.0-rc0-linux-i686.tar.gz
https://s3.amazonaws.com/julialang/bin/osx/x64/0.5/julia-0.5.0-rc0-osx10.7+.dmg
https://s3.amazonaws.com/julialang/bin/winnt/x64/0.5/julia-0.5.0-rc0-win64.exe
https://s3.amazonaws.com/julialang/bin/winnt/x86/0.5/julia-0.5.0-rc0-win32.exe
https://s3.amazonaws.com/julialang/bin/checksums/julia-0.5.0-rc0.sha256
https://s3.amazonaws.com/julialang/bin/checksums/julia-0.5.0-rc0.md5
(and append .asc
for gpg signatures of linux tarballs)
Announced on julia-users:
https://groups.google.com/forum/#!topic/julia-users/shrnNHpyliA
https://github.com/JuliaLang/julia/issues/17364 has the advantage of an open PR at #17399, we know what the fix is but need to make sure the test for it is representative (i.e. the test would fail without the fix) so it doesn't regress. I think https://github.com/JuliaLang/julia/issues/17610 actually has more serious consequences in regular use since it prevents packages from being updated or removed in normal use, but we haven't fully debugged that one or come up with a fix yet.
I think doing RC's on a time-based schedule for the next couple of weeks, one per week, makes the most sense. We'll get as many bugfixes and cleanups in as we can, and make a judgement call when we get ready for the next RC whether or not it's ready to be called 0.5.0 final. RC1 definitely is not, but if we tag it on Monday or Tuesday then it'll have a few more bugs fixed than RC0.
Apologies for being late to the party, but I'd like to see #17109/#17110 addressed in 0.5 (they are regressions from 0.4).
@jballanc: those are not going to be show stoppers for the 0.5 release but I've tagged them 0.5.x.
Yeah, I don't know whether we'll have time to fix those for 0.5.0 but they should have at least gotten triaged properly, probably got lost during JuliaCon.
@ViralBShah Could you indicate in your announcement emails to the larger user base whether a given RC is actually a candidate for a release or not? For example, for RC1, just add a sentence a la "We tagged RC1. RC1 is not a candidate for release because it has known bugs and there will be at least a RC2" or something like that. And then when there is an RC that you guys actually think is done, obviously also indicate that in the email. That sort of information really, really helps us people downstream.
The main reason to mention the URL to this page was to signal exactly that "this is mostly ready, but there are still a couple small issues - but we want you guys to give it a try".
The email could certainly be clearer about this.
fixing broken packages! see http://pkg.julialang.org/pulse.html, there's a lot of work to do to bring the ecosystem to a state where it works as well on 0.5 as it does on 0.4 right now
Is there a separate discussion around this, possibly for divvying up work for those who would be willing to update others packages?
Also, when I look at http://pkg.julialang.org/ there are a lot of packages that had their last 0.5 test result from a couple of months ago. Might make sense to run these package tests for all packages on RC0, so that the pass/fail results reflect how packages fare on the RC0 build, and not a many months old 0.5 build?
I think the testing status is up to date, but it's only testing the latest tagged release, which is what the date is.
@Keno Ah, you are right, I misinterpreted the labels there.
Was hoping we could get https://github.com/JuliaLang/julia/issues/17364 resolved over the weekend, but that moved a little slowly. I think if https://github.com/JuliaLang/julia/pull/17773, https://github.com/JuliaLang/julia/pull/17777, and https://github.com/JuliaLang/julia/pull/17779 get reviewed and look ready to merge then we'll branch and tag rc1 today.
We have branched. https://github.com/JuliaLang/julia/pull/17389 was a big performance regression, not entirely fixed by https://github.com/JuliaLang/julia/pull/17798, so I'm going to propose reverting it on the release-0.5 branch and making rc1 from that right now.
0.5.0-rc1 was tagged, with one more reversion because #17522 is causing test failures on Windows. Should have branched from before that was merged, but if the test failures can be resolved then we can consider reverting the revert and backporting the test fix for rc2.
Binaries are building now on the buildbots and will be ready soon. I'll refresh my memory on how to do the Windows code signing this time, then announce on the mailing list once everything is moved around to the julialang bucket on S3. The links will be the same, just replace rc0 with rc1.
Is there a separate discussion around this, possibly for divvying up work for those who would be willing to update others packages?
Not yet, and I'm not sure whether this repo is the best place to do so. Mailing list? Gitter? Placeholder repo in some organization?
A decent number of packages may be working okay on master but need tags, so in those cases some prodding to release a new version that passes on 0.5 may be in order. It wouldn't be too hard to modify PkgEval to test against master of everything if someone's interested in trying that and seeing how the results differ.
@randyzwitch as Tony Kelman said, for most largely used packages (PyPlot, Juno, all of JuliaStats, etc.) you'll probably notice that people already have it working on v0.5-rc0 but you just need to checkout master (likely on many dependencies as well). It would be best to work it out via Gitter or however else you can get in contact with the main developers for this reason. For example, to get Juno working on v0.5-rc0, you just have to checkout master on like 8-9 different repos. So the issue is mostly about tagging the new versions.
The only major issue would be to get everyone to root out the deprecations. If you use something like PyPlot, you get a wall of deprecation warnings, usually the string deprecations. It works but is off-putting. Someone should use PkgEval test and go around to fix the deprecations in the large projects, or at least the ones that many other packages depend on.
Just want to make it very clear that I echo @davidanthoff 's request above.
https://github.com/JuliaLang/julia/issues/17418#issuecomment-236679744
As an update here, I'm preparing a branch full of cherry-picks of backport pending PR's for RC2, which I'll be opening a PR for when it's ready to run nanosoldier and pkgeval on. (It will look a lot like master, minus a handful of things that aren't backport candidates.)
edit: #17960
rc2 will be tagged momentarily and uploaded as soon as it's built has been tagged and uploaded, just need to announce
Is rc2 actually literally a release candidate, unlike rc0 which was understood not to be, but more like a beta release?
rc2 is missing one major thing at the moment, which is proxy support - that exists on master but the build system with curl is rough around the edges and will need some work to get backported into rc3.
I'd like to nominate #17146 (or #17995, they might be dups) for 0.5.0. They corrupt repo state and I don't think 0.5.0 should ship with bugs on such a common code path that messes up local repositories.
RC3 backports are at #18156. This includes curl and should work with proxies, so will be the first release candidate that we could seriously consider promoting to 0.5.0 final. We'll decide after between a few days and a week of testing on RC3 whether to put out an RC4 or go straight to final.
See #18276 for RC4 backports. It doesn't look like many more bugfixes are going to happen right now, so anything that doesn't have a fix materialize soon will be 0.5.1 material.
I am wondering, should the issue described in #18353 get a NEWS mention (in short: non-obvious behaviour when passing redefined functions to map
, quadgk
etc. due to #265)?
I ran into this the moment I used 0.5 for interactive numeric work, and it took a moment to figure out why things were behaving strangely, since it works without problems on 0.4.
There doesn't seem to be a step in the release process for adding release-0.5
to readthedocs.org.
At the moment on docs.julialang.org you have versions release-0.3
, release-0.4
, stable
and latest
. For me it's not clear which one is currently for v0.5. Thank you.
Yes, good question @Ken-B. I have never touched readthedocs myself, @StefanKarpinski who has admin access there to create a new release option?
I now have admin access and enabled the release-0.5 branch and the 0.5 docs are live: http://docs.julialang.org/en/release-0.5/
What needs to be done for the stable
branch on readthedocs?
final few backports are at https://github.com/JuliaLang/julia/pull/18546
Is there going to be a RC5 for this stuff?
I'll post links to the binaries that are going to run through pkgeval, but the diff there is pretty small inside src and base.
will be tagging, building and uploading today
Most helpful comment
will be tagging, building and uploading today