stack build is killed after using up 4GB of memory

Created on 14 Mar 2016  Â·  8Comments  Â·  Source: commercialhaskell/stack

I'm running stack build on CircleCI, and after around 20 minutes, it's killed because it ate up the 4GB of memory available to the build container. I was unable to find a flag to set a memory limit. Is such a flag available? A cabal flag I could reuse, perhaps?

Here is the output:

Failing command: stack build --only-dependencies 
Exit code: -1 
Output: 
       socks-0.5.4: install
Progress: 60/125                asn1-parse-0.9.4: install
Progress: 60/125                Progress: 61/125                conduit-1.2.6: build
Progress: 61/125                Progress: 62/125                conduit-1.2.6: install
Progress: 62/125                Progress: 63/125                hs-bibutils-5.5: install
Progress: 63/125                Progress: 64/125                regex-pcre-builtin-0.94.4.8.8.35: install
Progress: 64/125                Progress: 65/125                cryptonite-0.6: install
Progress: 65/125                Progress: 66/125                x509-1.6.3: download
Progress: 66/125                x509-1.6.3: configure
Progress: 66/125                x509-1.6.3: build
Progress: 66/125                x509-1.6.3: install
Progress: 66/125                Progress: 67/125                x509-store-1.6.1: download
Progress: 67/125                x509-store-1.6.1: configure
Progress: 67/125                x509-store-1.6.1: build
Progress: 67/125                x509-store-1.6.1: install
Progress: 67/125                Progress: 68/125                x509-system-1.6.1: download
Progress: 68/125                x509-system-1.6.1: configure
Progress: 68/125                x509-validation-1.6.3: download
Progress: 68/125                x509-system-1.6.1: build
Progress: 68/125                x509-validation-1.6.3: configure
Progress: 68/125                x509-validation-1.6.3: build
Progress: 68/125                x509-system-1.6.1: install
Progress: 68/125                Progress: 69/125                x509-validation-1.6.3: install
Progress: 69/125                Progress: 70/125                tls-1.3.2: download
Progress: 70/125                tls-1.3.2: configure
Progress: 70/125                tls-1.3.2: build
Progress: 70/125                SHA-1.6.4.2: install
Progress: 70/125                Progress: 71/125                text-1.2.1.3: install
Progress: 71/125                Progress: 72/125                vector-0.10.12.3: install
Progress: 72/125                Progress: 73/125                tls-1.3.2: install
Progress: 73/125                Progress: 74/125                tagsoup-0.13.5: download
Progress: 74/125                zip-archive-0.2.3.7: download
Progress: 74/125                cmark-0.4.1: download
Progress: 74/125                fsnotify-0.2.1: download
Progress: 74/125                xml-types-0.3.6: download
Progress: 74/125                hashable-1.2.3.3: download
Progress: 74/125                system-filepath-0.4.13.4: download
Progress: 74/125                zip-archive-0.2.3.7: configure
Progress: 74/125                mime-types-0.1.0.6: download
Progress: 74/125                blaze-builder-0.4.0.1: download
Progress: 74/125                xml-1.3.14: download
Progress: 74/125                enumerator-0.4.20: download
Progress: 74/125                parsec-3.1.9: download
Progress: 74/125                JuicyPixels-3.2.6.4: download
Progress: 74/125                connection-0.2.5: download
Progress: 74/125                zip-archive-0.2.3.7: build
Progress: 74/125                tagsoup-0.13.5: configure
Progress: 74/125                tagsoup-0.13.5: build
Progress: 74/125                xml-types-0.3.6: configure
Progress: 74/125                xml-types-0.3.6: build
Progress: 74/125                fsnotify-0.2.1: configure
Progress: 74/125                fsnotify-0.2.1: build
Progress: 74/125                cmark-0.4.1: configure
Progress: 74/125                cmark-0.4.1: build
Progress: 74/125                hashable-1.2.3.3: configure
Progress: 74/125                hashable-1.2.3.3: build
Progress: 74/125                system-filepath-0.4.13.4: configure
Progress: 74/125                system-filepath-0.4.13.4: build
Progress: 74/125                mime-types-0.1.0.6: configure
Progress: 74/125                hashable-1.2.3.3: install
Progress: 74/125                mime-types-0.1.0.6: build
Progress: 74/125                blaze-builder-0.4.0.1: configure
Progress: 74/125                Progress: 75/125                fsnotify-0.2.1: install
Progress: 75/125                case-insensitive-1.2.0.5: download
Progress: 75/125                scientific-0.3.4.4: download
Progress: 75/125                unordered-containers-0.2.5.1: download
Progress: 75/125                Progress: 76/125                blaze-builder-0.4.0.1: build
Progress: 76/125                xml-1.3.14: configure
Progress: 76/125                xml-1.3.14: build
Progress: 76/125                enumerator-0.4.20: configure
Progress: 76/125                enumerator-0.4.20: build
Progress: 76/125                JuicyPixels-3.2.6.4: configure
Progress: 76/125                JuicyPixels-3.2.6.4: build
Progress: 76/125                connection-0.2.5: configure
Progress: 76/125                connection-0.2.5: build
Progress: 76/125                parsec-3.1.9: configure
Progress: 76/125                zip-archive-0.2.3.7: install
Progress: 76/125                parsec-3.1.9: build
Progress: 76/125                case-insensitive-1.2.0.5: configure
Progress: 76/125                Progress: 77/125                case-insensitive-1.2.0.5: build
Progress: 77/125                scientific-0.3.4.4: configure
Progress: 77/125                scientific-0.3.4.4: build
Progress: 77/125                unordered-containers-0.2.5.1: configure
Progress: 77/125                connection-0.2.5: install
Progress: 77/125                Progress: 78/125                unordered-containers-0.2.5.1: build
Progress: 78/125                blaze-builder-0.4.0.1: install
Progress: 78/125                Progress: 79/125                case-insensitive-1.2.0.5: install
Progress: 79/125                Progress: 80/125                blaze-markup-0.7.0.3: download
Progress: 80/125                cookie-0.4.1.6: download
Progress: 80/125                blaze-markup-0.7.0.3: configure
Progress: 80/125                streaming-commons-0.1.15: download
Progress: 80/125                http-types-0.8.6: download
Progress: 80/125                blaze-markup-0.7.0.3: build
Progress: 80/125                cookie-0.4.1.6: configure
Progress: 80/125
stack build --only-dependencies died unexpectedly

Action failed: stack build --only-dependencies
Failing command: Collect test metadata 
Exit code: 
Output: 


Action failed: Collect test metadata
The rest of your commands were successful: 
• Starting the build
• Start container
• Enable SSH
• Restore source cache
• Checkout using deploy key: 14:e9:82:d4:54:49:c6:9a:15:e2:72:54:6c:be:6c:54
• Configure the build
• sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 20
• sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.6 20
• sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 10
• sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.9 10
• Restore cache
• wget https://github.com/commercialhaskell/stack/releases/download/v0.1.3.1/stack-0.1.3.1-x86_64-linux.gz -O /tmp/stack.gz
• gunzip /tmp/stack.gz && chmod +x /tmp/stack
• sudo mv /tmp/stack /usr/bin/stack
• stack setup +RTS -M1024m -K1024m
• Collect artifacts
• Disable SSH
picture beerendlauwers
👍1

Most helpful comment

There is no way to set an overall memory limit. stack build -j1 should help, because then it won't run builds in parallel, but this will make the build take longer. Does CircleCI have caching? Could probably benefit a lot from that.

Related issues: https://github.com/commercialhaskell/stack/issues/773 https://github.com/commercialhaskell/stack/issues/859

picture mgsloan on 14 Mar 2016
👍6

All 8 comments

There is no way to set an overall memory limit. stack build -j1 should help, because then it won't run builds in parallel, but this will make the build take longer. Does CircleCI have caching? Could probably benefit a lot from that.

Related issues: https://github.com/commercialhaskell/stack/issues/773 https://github.com/commercialhaskell/stack/issues/859

mgsloan picture mgsloan on 14 Mar 2016
👍6

Thanks, stack build -j1 lowered the memory use to acceptable levels.

beerendlauwers picture beerendlauwers on 15 Mar 2016

By curiosity, is there a way to specify this in stack.yaml ?

ThomasCrevoisier picture ThomasCrevoisier on 15 Nov 2017

@ThomasCrevoisier Yes, there is. jobs: 1. Seems there are some gaps in the configuration documentation!

mgsloan picture mgsloan on 15 Nov 2017

Perfect, I'll try that. Thanks a lot ! :)

ThomasCrevoisier picture ThomasCrevoisier on 16 Nov 2017

@ThomasCrevoisier Welcome! I've opened the PR at #3581 to add docs for this and other options that were missing.

mgsloan picture mgsloan on 16 Nov 2017

Aaaaaah, I arrive too late :P I planned to patch the docs, but you were to quick :)
Anyway, this looks good :) Thanks a lot !

ThomasCrevoisier picture ThomasCrevoisier on 16 Nov 2017

@ThomasCrevoisier Oh, awesome! Feel free to make such PRs in the future, much appreciated :)

mgsloan picture mgsloan on 16 Nov 2017
Was this page helpful?
0 / 5 - 0 ratings

Related issues

igrep picture igrep  Â·  3Comments
rrnewton picture rrnewton  Â·  4Comments
Toxaris picture Toxaris  Â·  4Comments
s5k6 picture s5k6  Â·  3Comments
sjakobi picture sjakobi  Â·  4Comments