nightly pip wheel builds were building successfully until December 30th. On December 31st the builds started failing for these build flavors:
cu90
cu100
cu101
cu90mkl
cu100mkl
cu101mkl
All builds failed with the same error message:
/usr/bin/ld: /root/pip_build/mxnet-build/staticdeps/usr/lib/x86_64-linux-gnu/libnccl.a(init.o): unrecognized relocation (0x2a) in section `.text'
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make: *** [lib/libmxnet.so] Error 1
make: *** Waiting for unfinished jobs....
/usr/bin/ld: /root/pip_build/mxnet-build/staticdeps/usr/lib/x86_64-linux-gnu/libnccl.a(transport.o): unrecognized relocation (0x2a) in section `.text.startup'
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make: *** [bin/im2rec] Error 1
Build using the docker container used in the CI:
mxnetci/publish.ubuntu1404_cpu
./tools/staticbuild/build.sh $mxnet_variant pip
@mxnet-label-bot update [Build]
working on it in https://github.com/szha/mxnet/tree/fix_cd_s3 (https://github.com/szha/mxnet/commit/f2035bf4b2d282e8fd5344e4afddcc6f21cd7fc1)
fixed in #17259
Most helpful comment
working on it in https://github.com/szha/mxnet/tree/fix_cd_s3 (https://github.com/szha/mxnet/commit/f2035bf4b2d282e8fd5344e4afddcc6f21cd7fc1)