Julia: startup time

[ghost]

~~~sh
$ julia -v
julia version 0.7.0-beta

$ time julia -e 'println("hello world")'
hello world
real 0m0.406s
~~~

Compare with Perl:

~sh
$ time perl -e 'print("hello world")'
hello world
real 0m0.016s
~

Or Python:

~sh
$ time python3 -c 'print("hello world")'
hello world
real 0m0.047s
~

Or PHP:

~sh
$ time php -r 'echo "hello world";'
hello world
real 0m0.047s
~

Or Ruby:

~sh
$ time ruby -e 'puts "hello world"'
hello world
real 0m0.187s
~

[JeffBezanson]

This is an issue with startup time, or more generally compile time, and not performance (in the sense of throughput) in general. We are well aware of it; you can find more examples under the "latency" issue label. Work on this is ongoing, though currently we have to prioritize addressing the much longer delays that can happen when loading certain packages.

[JeffBezanson]

This issue has a lot of history; see for example #260 #1064 #7280 #4373 #4452 . We have actually improved significantly over the years on this metric, and hopefully will continue to do so.

[ghost]

@JeffBezanson it seems devs here are "close happy"?

every single one of those issues you linked have been closed, and i would consider this issue far from fixed.

julia is currently and order of magnitude (or more) slower than other popular languages.

[JeffBezanson]

I didn't say the issue was fully resolved, I said it had a lot of history. For example in one of those issues the startup time was over 2 seconds, so perhaps we considered it fixed relative to that. We will continue to work on this. Please be patient.

[KristofferC]

Doing something like https://github.com/JuliaLang/julia/pull/28075 would approximately half current startup time.

https://asciinema.org/a/hMmHsdvVxqhQTkcO60uz1c23U for that PR vs status quo.

[pablosanjose]

Not to understate the issue here, but although Julia can indeed be used as a quick sripting language (I myself collect arXiv preprints using a quick Julia script), many of the most relevant applications of the language involve heavy computations, for which 0.5 secs is peanuts. I myself am already quite pleased with startup latency relative to the old times, and if @JeffBezanson says so, you can count on further improvements in that regard.

[ufechner7]

Well, currently the startup time of 0.7 is higher than for 0.6. It would be nice if that could be fixed.
Just to make Julia 0.7 even more awesome! :)

[JeffBezanson]

Update: on my system this has now gone from about 0.5 sec to 0.2.

[ufechner7]

On my system Julia 0.7 is still significantly slower than 0.6:

ufechner@TUD277255:~$ time julia7 -e 'println("hello world")'
hello world

real    0m0.218s
user    0m0.248s
sys 0m0.206s
ufechner@TUD277255:~$ time julia -e 'println("hello world")'
hello world

real    0m0.170s
user    0m0.172s
sys 0m0.112s
ufechner@TUD277255:~$ 

Version info:

ufechner@TUD277255:~$ julia7
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-beta2.20 (2018-07-16 18:40 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit fbc30cd* (0 days old master)
|__/                   |  x86_64-linux-gnu 

[KristofferC]

Can you try build https://github.com/JuliaLang/julia/pull/28118 and report the results.

[pablosanjose]

Julia v0.7beta.152

$ time julia -e 'println("hello world")'
hello world

real    0m0.478s
user    0m0.332s
sys 0m0.129s

Julia v0.7beta2.26

$ time ./julia -e 'println("hello world")'
hello world

real    0m0.237s
user    0m0.137s
sys 0m0.105s

In the #28118 branch

$ time ./julia -e 'println("hello world")'
hello world

real    0m0.266s
user    0m0.138s
sys 0m0.131s

Julia v0.6.1

$ time /Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -e 'println("hello world")'
hello world

real    0m0.338s
user    0m0.246s
sys 0m0.111s

EDIT: on a macbook pro with an Intel i7-7567U CPU @ 3.50GHz
EDIT2: added current beta2. This promises to be even better once #28118 is merged!

[tknopp]

28118 is not rebased on master, right? So then we might get below 100 ms, which seems to be a massive improvement.

[KristofferC]

There is probably overlap that #28118 precompiled functions that are now instead lazily executed. I haven't checked how big that overlap is.

[tknopp]

I see, #28118 is already rebased.

[KristofferC]

(I just pushed a rebased version)

[pablosanjose]

Rebased #28118 (best time of several)

$ time ./julia -e 'println("hello world")'
hello world

real    0m0.238s
user    0m0.127s
sys 0m0.115s

Does that make sense??

[KristofferC]

If almost all precompileoverhead from starting julia has been removed then that makes sense and #28118 is only good for making the REPL snappier.

[pablosanjose]

I can confirm that #28118 is still much snappier than master, despite both launching equally fast. The ?reinterpret test still shows four distinguishable blocks of text appearing on master, while on #28118 it's instantaneous. I have to say, however, that I see a very noticeable improvement in master relative to an older beta.

[ufechner7]

I can confirm that Julia 0.7 is now faster than Julia 0.6.4:

ufechner@TUD277255:~/julia$ time julia7 -e 'println("hello world")'
hello world

real    0m0.156s
user    0m0.149s
sys 0m0.116s
ufechner@TUD277255:~/julia$ time julia -e 'println("hello world")'
hello world

real    0m0.166s
user    0m0.175s
sys 0m0.122s
ufechner@TUD277255:~/julia$ 

Julia version used:

ufechner@TUD277255:~/julia$ julia7
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-beta2.29 (2018-07-17 17:33 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit ee7e23d* (0 days old master)
|__/                   |  x86_64-linux-gnu

Good job! :+1:

[ghost]

@ufechner7 thanks - it looks like it might be time to close this - as it appears julia is even faster than ruby now - i dont see any artifacts here

http://ci.appveyor.com/project/JuliaLang/julia

so looks like i will need to do a build myself before i can test this

[ufechner7]

The assumption, that Julia would start faster than ruby is unfortunately wrong. It is still about five times slower. But from my point of view that's good enough for now, taken the huge runtime performance benefits into account.

ufechner@TUD277255:~$ time ruby -e 'puts "hello world"'
hello world

real    0m0.032s
user    0m0.027s
sys 0m0.004s
ufechner@TUD277255:~$

I just have a fast computer. Nevertheless, from my point of view Julia 0.7 is already fast enough to be released. Further latency improvements can happen later.

[oscardssmith]

One thing worth noting is that at this point, we should probably not be measuring print time in start up as printing is kind of slow. Something like '1+1' might be a better test.

[ufechner7]

You are right, in the 1+1 contest Julia 0.7 performs worse, it is about 7 times slower than ruby and even slower than Julia 0.6.4. And yes, without println performance improves, but I think it is more realistic to print a result than not.

ufechner@TUD277255:~$ time julia7 -e 'println(1+1)'
2

real    0m0.212s
user    0m0.263s
sys 0m0.328s
ufechner@TUD277255:~$ time ruby -e 'puts 1+1'
2

real    0m0.032s
user    0m0.029s
sys 0m0.003s
ufechner@TUD277255:~$ time julia -e 'println(1+1)'
2

real    0m0.178s
user    0m0.191s
sys 0m0.210s

So still room for improvement!

[StefanKarpinski]

I would just time [julia|ruby|perl] -e0 and python -c0, doing an operation is unnecessary.

[KristofferC]

Dup of https://github.com/JuliaLang/julia/issues/17285. Don't think we need to have both open.

[ghost]

@KristofferC how can my issue be a duplicate of... my issue?

[KristofferC]

Oh, mistake :) https://github.com/JuliaLang/julia/issues/17285 (updated post above)