Pandas: gcc crashed when building pandas

Created on 10 Sep 2012 · 24Comments · Source: pandas-dev/pandas

I experienced a gcc crash when building the latest pandas.

This was on Ubuntu 12.04 gcc version 4.6, Cython 0.17.

On Ubuntu 11.10 gcc version 4.6.1, Cython 0.14 I did not have the issue.

Build

Source

codeape2

Most helpful comment

Setting up a temporary swapfile before building pandas helps.

I managed to compile with 512 MB swap created with the following commands:

sudo dd if=/dev/zero of=/swapfile bs=1024 count=524288
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

wuan on 30 Oct 2012

👍12

All 24 comments

Was the crash consistent? Or appear at random?

wesm on 10 Sep 2012

fyi Ubuntu 12.04 + gcc 4.6.3 + cython 0.16 works fine for me (did not try with cython 0.17)

(pandas)[lodagro@ubuntu][515][i]  lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.1 LTS
Release:    12.04
Codename:   precise
(pandas)[lodagro@ubuntu][516][i] cython --version
Cython version 0.16
(pandas)[lodagro@ubuntu][516][i] gcc --version
gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

lodagro on 10 Sep 2012

The crash was consistent.

codeape2 on 10 Sep 2012

Any specific error messages?

changhiskhan on 10 Sep 2012

Steps to reproduce:

Amazon EC2 t1.micro instance ubuntu/images/ebs/ubuntu-precise-12.04-amd64-server-20120616 (ami-ab9491df)

First try:

Python, python-dev, build-essentials, python-pip, python-virtualenv installed from Ubuntu packages
Numpy and Cython installed using pip

python setup.py build

gives error:

In file included from pandas/src/tseries.c:260:0:
pandas/src/numpy_helper.h:47:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
pandas/src/tseries.c: In function ‘__pyx_f_6pandas_3lib__bin_search’:
pandas/src/tseries.c:120278:13: warning: ‘__pyx_v_mid’ may be used uninitialized in this function [-Wuninitialized]
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
error: command 'gcc' failed with exit status 4

Full console log: http://pastebin.com/eNjwjP8d

$ gcc -dumpversion; cython -V
4.6
Cython version 0.17

Second try:

Cython from Ubuntu package

No errors, so it seems Cython is the problem.

$ gcc -dumpversion; cython -V
4.6
Cython version 0.15.1

So it seems that...

Cython 0.17 has a problem that Cython 0.15.1 does not have.

codeape2 on 10 Sep 2012

Builds cleanly for me with gcc-4.6.1 and Cython 0.17 =/

wesm on 11 Sep 2012

Builds fine for me with Cython 0.17 / Ubuntu 12.04.1.

$ uname -a
Linux highorder 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.1 LTS
Release:    12.04
Codename:   precise
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.6.3-1ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)

Given that the compiler _crashed_ this is unfortunately an issue with gcc or your environment, not with pandas. Nonetheless it might be possible that a change here could alleviate your problem.

Maybe you ran out of memory on the t1.micro instance? I don't believe they come with swap space by default. Compiling pandas/src/tseries.c used ~418MB for me.

bfroehle on 11 Sep 2012

That might be. that extension is very memory hungry to build

wesm on 11 Sep 2012

Yes, looks like memory was the problem. It works on a m1.small instance.

codeape2 on 11 Sep 2012

Cool. I guess on t1.micro you would want to compile pandas elsewhere and download the binary; alternately if some brave soul wants to help refactor the Cython code into smaller modules to reduce the memory footprint of gcc...

wesm on 11 Sep 2012

I had this same problem trying to build pandas on an Amazon EC2 t1.micro instance using:

$ easy_install pandas

...
pandas/src/tseries.c:120278:13: warning: ‘__pyx_v_mid’ may be used uninitialized in this function [-Wuninitialized]
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
...

To work around it I followed the instructions for creating an AMI of the current t1.micro instance (following these directions: http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/Tutorial_CreateImage.html), I then created an new m1.large instance from the AMI. Once I loaded that new instance I again tried:

$ easy_install pandas

This time I had no problems now that I had access to more memory in the large instance.

After the installation of pandas I again created an AMI of the improved m1.large instance (with pandas now compiled) and used this improved AMI to build a NEW t1.micro instance with pandas.

The whole process took less than half an hour, but it was a pain to figure out how to deal with the gcc crash in the first place.

amills61 on 30 Oct 2012

I did something similar. I installed on a normal instance then moved the site-packages/pandas directory to the micro instance.

codeape2 on 30 Oct 2012

Unfortunately compiling pandas's C extensions requires a lot of RAM (about half a gig or so). It would be nice if gcc gave a more informative error message

wesm on 30 Oct 2012

Setting up a temporary swapfile before building pandas helps.

I managed to compile with 512 MB swap created with the following commands:

sudo dd if=/dev/zero of=/swapfile bs=1024 count=524288
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

wuan on 30 Oct 2012

👍12

Just confirming I had this issue and creating a temporary swapfile worked for me. Thanks wuan.

mattlangeman on 29 Nov 2012

FWIW I'm planning some refactoring that should reduce the build memory footprint (hopefully)

wesm on 29 Nov 2012

The solution from wuan worked for me too. Thank you.

javpaw on 9 Dec 2012

Wuan's solution worked for me too! Thank you!!

seidapps on 11 Mar 2014

Thank you Wuan!! Worked for me!

fproldan on 29 May 2014

@wuan thanks for the tip, that helped me here as well.

mikeatlas on 15 Jul 2014

Just a finally command after you install do you pandas build and are using a temp swap as recommended by @wuan, remove the swap with sudo swapoff /swapfile

fission6 on 11 Aug 2014

👍2

Has a similar issue. Thanks @wuan :) . Saved me :)

sandeepan on 7 Sep 2015

I got this error message after using @wuan solution.

dd: failed to open '/swapfile': Text file busy

How to fix it?

datomnurdin on 14 Nov 2015

@datomnurdin it's seems you already have swapfile, you need unmont current sudo swapoff -a and create new one. Or maybe your swap is fully, you can try to restart machine and rebuild pandas again.