Incubator-mxnet: updated NCCL version breaks nightly pip wheel builds

Created on 9 Jan 2020  路  3Comments  路  Source: apache/incubator-mxnet

Description

nightly pip wheel builds were building successfully until December 30th. On December 31st the builds started failing for these build flavors:
cu90
cu100
cu101
cu90mkl
cu100mkl
cu101mkl

Error Message

All builds failed with the same error message:

/usr/bin/ld: /root/pip_build/mxnet-build/staticdeps/usr/lib/x86_64-linux-gnu/libnccl.a(init.o): unrecognized relocation (0x2a) in section `.text' 
/usr/bin/ld: final link failed: Bad value 
collect2: error: ld returned 1 exit status 
make: *** [lib/libmxnet.so] Error 1 
make: *** Waiting for unfinished jobs.... 
/usr/bin/ld: /root/pip_build/mxnet-build/staticdeps/usr/lib/x86_64-linux-gnu/libnccl.a(transport.o): unrecognized relocation (0x2a) in section `.text.startup' 
/usr/bin/ld: final link failed: Bad value 
collect2: error: ld returned 1 exit status 
make: *** [bin/im2rec] Error 1 

To Reproduce

Build using the docker container used in the CI:
mxnetci/publish.ubuntu1404_cpu

./tools/staticbuild/build.sh $mxnet_variant pip
Build

Most helpful comment

working on it in https://github.com/szha/mxnet/tree/fix_cd_s3 (https://github.com/szha/mxnet/commit/f2035bf4b2d282e8fd5344e4afddcc6f21cd7fc1)

All 3 comments

@mxnet-label-bot update [Build]

working on it in https://github.com/szha/mxnet/tree/fix_cd_s3 (https://github.com/szha/mxnet/commit/f2035bf4b2d282e8fd5344e4afddcc6f21cd7fc1)

fixed in #17259

Was this page helpful?
0 / 5 - 0 ratings