Stack: Default configJobs value is unsafe/wrong

[hasufell]

The default configJobs value is:

     configJobs <-
        case getFirst configMonoidJobs of
            Nothing -> liftIO getNumProcessors
            Just i -> return i

This just counts your number of processors and assumes you have

1. enough ram to run (nproc) jobs
2. fast enough hard drive (IO) to run (nproc) jobs
3. actually enough CPU power to not slow down everything else you are currently doing

As such, the default value is unsafe. It crashed my laptop today with 8 cores and 16GB of ram on installing pandoc, leaving unsaved files in nirvana. Luckily, there was no corruption.

Having a useful heuristics to determine a default value for jobs is hard. Many different things play together when you compile, not just number of processors, which is the most naive assumption. As such, the default value should be 1 and nothing else.

This also is in line with the stack slogan "Stack just works". If it crashes my laptop, it certainly isn't.

[mihaimaruseac]

I see some options to go forward:

1. default to 1 job
2. add flag and environment variable to get more than one parallel job

Do you want to make PRs for this?

Thanks

[hasufell]

add flag and environment variable to get more than one parallel job

What exactly do you have in mind here? If people want number of processor jobs, they can just do --jobs=$(nproc), right?

And I doubt anyone would actually want to do this, since the "jobs = number of processors" thing really comes from the C world, which has completely different memory footprint during compilation.

[mihaimaruseac]

I have several projects where a large set of files is compiled and would greatly benefit from specifying number of jobs.

Now that I see that there is -j/--jobs option I'm curious whether we should make default be 1 job and let everyone who wants more parallel compilation (almost everyone imho) specify jobs or keep default as is (or make slightly clever) and let the few people who want only 1 job specify -j1?

[hasufell]

You cannot really know what people want. It's just guessing... about what they want, what their projects look like, what system specs they have, ...

Defaults should be safe, not be based on assumptions and not try to be too smart. -j1 is exceptionally safe. And as demonstrated, system specs with 8 cores and 16gb are probably quite common and stack WILL crash/freeze these, depending on what packages you install. The OOM killer will not help you, you will have to use earlyoom... and that's not really something you want to tell someone who uses stack with default settings, right?

[snoyberg]

Given how many requests we've had to have more parallelism, the change you're requesting would be highly unpopular. Closing as wontfix, especially since it's possible to change the setting quite easily