Docker-stacks: Update to Julia 0.6

[appleparan]

If you are requesting a library upgrade or addition in one of the existing images, please state the desired library name and version here and disregard the remaining sections.

• Please update Julia kernel to v0.6. It's been a 2 months after Julia v0.6 have been released. I think it's time to update Julia kernel to v0.6 because it introduce new type system and some breaking changes (changelog).

[naveenkumarmarri]

@parente I can try this out
Can you point me which files to change?

[parente]

Julia is installed from https://launchpad.net/~staticfloat/+archive/ubuntu/juliareleases which currently has 0.5.2 as the latest available release. It's installed here in the datascience-notbook/Dockerfile: https://github.com/jupyter/docker-stacks/blob/master/datascience-notebook/Dockerfile#L22

I'm open to changing how we install Julia if that necessary. Or maybe that upstream channel could use help in packaging 0.6. I'm not a Julia user (Julian?) so contributions on how to move this forward are most welcome.

[jakirkham]

Related note, we have started building Julia in conda-forge. Haven't really added any related packages yet. Though could still try using the standard Julia mechanisms for installation with it.

[parente]

Somehow, I knew you'd be popping in, @jakirkham, I just didn't know how soon. 💯

https://github.com/conda-forge/julia-feedstock

@naveenkumarmarri If you want to give it a shot, try removing the install of julia using apt and run conda install julia instead. Then see if the various julia -e 'Pkg.init()' commands further down in the Dockerfile succeed or not.

If you start on the PR, and give maintainers access to contribute to it, we can iterate on it together.

[naveenkumarmarri]

@parente sure

[appleparan]

https://github.com/staticfloat/julia-docker/tree/master/julia

you can check these files for updating julia, it needs for julia-only docker images. It just download julia binaries from official homepage.

[naveenkumarmarri]

@parente I checked conda search julia --channel conda-forge
The version is 0.5.2
I couldn't find any package with 0.6 version
Should we install from source code?

[parente]

PR for Julia 0.6 opened for the conda-forge feedstock 2 hours ago.

https://github.com/conda-forge/julia-feedstock/pull/7

EDIT: Hit comment too soon. I suggest we help move that forward rather than go to source if possible. In the meantime, you could try installing the 0.5.2 from conda-forge and make sure the julia commands to install additional packages still work with it.

[tkelman]

Please use the upstream Julia binaries. It's a simple download and extracting a tarball on Linux. The conda build of julia should be considered experimental and is not a recommended configuration. The ppa is no longer maintained.

[jakirkham]

There is a 0.6.0 package for julia from conda-forge available. I leave it up to the community whether to use that or some other installation method.

[tkelman]

It has serious performance problems due to not being built with any optimizations. If there are any bugs caused by the unconventional way it's built, the response from upstream will be "use the official binaries."

[jakirkham]

It has serious performance problems...

This is not well demonstrated at this point. Only two build logs have been compared from two different CIs. One from conda-forge's CI and one from the Julia team's CI. Given CIs historically are resource limited and vary greatly in capabilities and configurations based on their provider, there are not really any conclusions to draw here.

To get an accurate answer, it will require someone (probably several someones) testing both packages in an apples-to-apples comparison to really know the truth about each one's performance. Would be happy to look at stock benchmarks and compare if there is interest. Feel free to discuss back at the feedstock's issue tracker.

...due to not being built with any optimizations.

It remains unclear to me what flags Julia already provides in its build system. From a cursory glance it looked like -O3 was hard coded in a few places regardless of what the user had in CFLAGS. Feel free to update with more details in the feedstock's issue tracker about additional flags that are needed.

If there are any bugs caused by the unconventional way it's built, the response from upstream will be "use the official binaries."

Please report issues about conda-forge packages' to their respective feedstocks'. This goes with all conda-forge packages (not just julia). If it was missed before, here is the julia feedstock's issue tracker. If the issue is determined to be an upstream one, it can then be raised upstream accordingly.

It is certainly upstream's prerogative to encourage use of their official binaries. Unfortunately not all users and use cases are created the same. So while it is one appropriate answer, it will not solve all problems equally well. Recognizing and addressing these differences accordingly will likely result in adoption of the language by a wider audience. 😉

---

Also if there are other concerns with how we are building it, the recipe is open sourced in this feedstock. Contributions welcome. 😄

[tkelman]

it will not solve all problems equally well

You have to admit the conda-forge build of Julia is far less widely used, untested with most packages and environments, and only supports a handful of the platforms that the upstream project already provides binaries for. What problems does that solve? You're adding a support and maintenance burden for both upstream and yourself (and any package author who may have to deal with issues caused by the unsupported way conda-forge is building things), without very compelling technical reasons for doing so.

Contributions welcome

Unless you would merge a contribution that replaced the recipe with downloading and using the official upstream binaries, this isn't true.