julia-1.2.0-full.tar.gz does not build behind firewall

[vmjersey]

As This worked fine when I downloaded and built Julia 1.0 from source. However, when I download the julia-1.2.0-full.tar.gz file from https://github.com/JuliaLang/julia/releases/download/v1.2.0/julia-1.2.0-full.tar.gz, I can't get it to build on an isolated network. Currently the steps I follow are:

% tar -xvf julia-1.2.0-full.tar.gz
% make
Warning: git information unavailable; versioning information limited
Warning: git information unavailable; versioning information limited
Warning: git information unavailable; versioning information limited
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to cache.julialang.org:443; Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to github.com:443; Connection refused
make[1]: * [/home/user1/julia-1.2.0/deps/srccache/SuiteSparse.v5.4.0-2.x86_64-linux-gnu-gcc4.tar.gz] Error 7
make: * [julia-deps] Error 2

OS: RHEL 7.6 fully patched.

Any help would be greatly appreciated.

[StefanKarpinski]

You may need to set USE_BINARYBUILDER to false when making. We switched to using pre-built dependencies in the meantime. That puts the full source tarball in a slightly weird situation since you will download the source for all dependencies but then by default it will ignore that source and try to build pre-build dependencies when building. We should certainly document that, but I wonder if we should maybe have USE_BINARYBUILDER default to false in the full source tarball.

[StefanKarpinski]

cc @staticfloat

[fredrikekre]

Duplicate of #33461

[Keno]

but I wonder if we should maybe have USE_BINARYBUILDER default to false in the full source tarball.

Seconded.

[vmjersey]

USE_BINARYBUILDER = 0 in the Make.user file seemed to fix the build issue. Thanks for the help.

[ViralBShah]

Might be a good issue for someone to submit a PR to the build system.

[StefanKarpinski]

Would the right approach be to ship julia-x.y.z-full.tar.gz with a default Make.user? Or should we try to patch the default in Make.inc when we generate the tarball?

[Keno]

I think shipping it with a default Make.user is the correct way to do this.

[StefanKarpinski]

Ok, so if someone wants to take this issue up, they should modify the make rule that generates julia-x.y.z-full.tar.gz to put a Make.user file that includes USE_BINARYBUILDER = 0.

[ViralBShah]

Technically Make.user is strictly for the user and not for system developers. I am not sure if we ever wrote that down, but that was certainly the intent. Ideally the Makefiles would detect which type of build it is and have rules to set the flags. It should be easy to test for the presence of certain files only in the full build.

In general one can delete the Make.user to get rid of local changes to get the default settings. Providing a default Make.user would make it difficult to do so.

[Keno]

The default settings are to use the BinaryBuilder binaries, so I don't see the inconsistency.

[nalimilan]

Why not just change the default value in Make.inc for 1.2.1? I don't get why we would ship Make.user...

[StefanKarpinski]

Because we want the default when people clone and build Julia to be to use BB versions of dependencies. That's been a _huge_ improvement both to the time to build Julia from scratch and to the reliability of our BB versions of all dependencies since using them is the default. Why would we want to go back to the bad old days of building all deps from source all the time? The only case where that seems like a good idea is when someone has downloaded a full source tarball, since that indicates that they want to build everything from source without downloading anything.

I do agree that shipping a Make.user file feels a bit strange, but that does seem like the simples way to easily affect the behavior of this particular tarball without affecting the default. It doesn't prevent people from further modifying the Make.user file.

[nalimilan]

I see. Though it is really worse to modify Make.inc before making the "full" tarball than adding Make.user? We could even add a command to modify Make.inc before calling tar in the Makefile.

[StefanKarpinski]

Sure, patching Make.inc when producing the full tarball was my suggestion.

[vtjnash]

Editing the files seems seems slightly worse to me. The purpose of Make.user is generally to apply custom patches—I don't see the conflict with this usage for it.