Itk: Build fails with `-DBUILD_TESTING=ON` in Gentoo

[TheChymera]

Steps to Reproduce

cmake -C /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_common_config.cmake -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON -DITK_USE_SYSTEM_DCMTK=ON -DGDCM_USE_SYSTEM_OPENJPEG=ON -DITK_USE_SYSTEM_DOUBLECONVERSION=ON -DITK_USE_SYSTEM_CASTXML=ON -DITK_USE_SYSTEM_HDF5=ON -DITK_USE_SYSTEM_JPEG=ON -DITK_USE_SYSTEM_PNG=ON -DITK_USE_SYSTEM_SWIG=ON -DITK_USE_SYSTEM_TIFF=ON -DITK_USE_SYSTEM_ZLIB=ON -DITK_BUILD_DEFAULT_MODULES=ON -DITK_COMPUTER_MEMORY_SIZE=1 -DWRAP_ITK_JAVA=OFF -DWRAP_ITK_TCL=OFF -Ddouble-conversion_INCLUDE_DIRS=/usr/include/double-conversion -DBUILD_TESTING=yes -DBUILD_EXAMPLES=no -DITK_USE_REVIEW=no -DITK_INSTALL_LIBRARY_DIR=lib64 -DITK_WRAP_PYTHON=OFF -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_toolchain.cmake /var/tmp/portage/sci-libs/itk-5.0_rc1/work/ITK-5.0rc01

Actual behavior

build.log

Versions

5.0_rc1

Environment

Gentoo Linux, cmake 3.11.4-r1

[phcerdan]

Thanks for the report @TheChymera, please post the ITK log file showing the error. From that emerge log, it seems located in /var/tmp/portage/sci-libs/itk-5.0_rc1/temp/build.log.

[phcerdan]

At first sight there are some inconsistencies in that CMake command.

-DBUILD_TESTING=yes -DBUILD_EXAMPLES=no

use instead

-DBUILD_TESTING:BOOL=ON -DBUILD_EXAMPLES:BOOL=OFF

Also:

-DCMAKE_BUILD_TYPE=Gentoo

Maybe that's handled by the Gentoo CMake toolchain provided, but that's CMAKE_BUILD_TYPE is not recognized by ITK for sure, try with Release.

Did you try to build ITKv4 in Gentoo? Maybe it is worth to ask in the Gentoo forum first.

[TheChymera]

@phcerdan you were reading the exact log you asked for. What do you mean by “try with Release”?

Both

cmake -C /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_common_config.cmake -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON -DITK_USE_SYSTEM_DCMTK=ON -DGDCM_USE_SYSTEM_OPENJPEG=ON -DITK_USE_SYSTEM_DOUBLECONVERSION=ON -DITK_USE_SYSTEM_CASTXML=ON -DITK_USE_SYSTEM_HDF5=ON -DITK_USE_SYSTEM_JPEG=ON -DITK_USE_SYSTEM_PNG=ON -DITK_USE_SYSTEM_SWIG=ON -DITK_USE_SYSTEM_TIFF=ON -DITK_USE_SYSTEM_ZLIB=ON -DITK_BUILD_DEFAULT_MODULES=ON -DITK_COMPUTER_MEMORY_SIZE=1 -DWRAP_ITK_JAVA=OFF -DWRAP_ITK_TCL=OFF -Ddouble-conversion_INCLUDE_DIRS=/usr/include/double-conversion -DBUILD_TESTING=ON -DBUILD_EXAMPLES=OFF -DITK_USE_REVIEW=OFF -DITK_INSTALL_LIBRARY_DIR=lib64 -DITK_WRAP_PYTHON=OFF -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_toolchain.cmake /var/tmp/portage/sci-libs/itk-5.0_rc1/work/ITK-5.0rc01

and

cmake -C /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_common_config.cmake -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON -DITK_USE_SYSTEM_DCMTK=ON -DGDCM_USE_SYSTEM_OPENJPEG=ON -DITK_USE_SYSTEM_DOUBLECONVERSION=ON -DITK_USE_SYSTEM_CASTXML=ON -DITK_USE_SYSTEM_HDF5=ON -DITK_USE_SYSTEM_JPEG=ON -DITK_USE_SYSTEM_PNG=ON -DITK_USE_SYSTEM_SWIG=ON -DITK_USE_SYSTEM_TIFF=ON -DITK_USE_SYSTEM_ZLIB=ON -DITK_BUILD_DEFAULT_MODULES=ON -DITK_COMPUTER_MEMORY_SIZE=1 -DWRAP_ITK_JAVA=OFF -DWRAP_ITK_TCL=OFF -Ddouble-conversion_INCLUDE_DIRS=/usr/include/double-conversion -DBUILD_TESTING:BOOL=ON -DBUILD_EXAMPLES:BOOL=OFF -DITK_USE_REVIEW=OFF -DITK_INSTALL_LIBRARY_DIR=lib64 -DITK_WRAP_PYTHON=OFF -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_toolchain.cmake /var/tmp/portage/sci-libs/itk-5.0_rc1/work/ITK-5.0rc01

fail with basically the exact same logs:
build.log

I don't know what -DCMAKE_BUILD_TYPE=Gentoo is all about, but it works well as long as -DBUILD_TESTING=ON is not set.
The otherwise latest ITK we provide to our users is 4.12.0, and that builds nicely, but a lot of stuff has had to be updated for this version bump (e.g. https://github.com/InsightSoftwareConsortium/ITK/issues/572 https://github.com/InsightSoftwareConsortium/ITK/issues/618 https://github.com/InsightSoftwareConsortium/ITK/issues/635).

[phcerdan]

It seems a problem downloading data for the tests.

make[2]: Entering directory '/var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build'
cd /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /var/tmp/portage/sci-libs/itk-5.0_rc1/work/ITK-5.0rc01 /var/tmp/portage/sci-libs/itk-5.0_rc1/work/ITK-5.0rc01/Modules/ThirdParty/GDCM/src/gdcm/Source/Common /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/Modules/ThirdParty/GDCM/src/gdcm/Source/Common /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/Modules/ThirdParty/GDCM/src/gdcm/Source/Common/CMakeFiles/gdcmCommon.dir/DependInfo.cmake --color=
-- Fetching "https://itk.org/files/ExternalData/MD5/5970c467ac9b44e55829ccf0a1296fd1"
-- Fetching "https://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5"
-- Fetching "https://insightsoftwareconsortium.github.io/ITKTestingData/SHA512/0aad3941e82b5cb077b791606abdb186c4284956f03b58a629f2434a98b5e4bf8efcc32a7aa66791b16c57cf4da5cb83d9801e2d7db3d0e58e1ada401ddbc20c"
-- Fetching "https://data.kitware.com:443/api/v1/file/hashsum/SHA512/0aad3941e82b5cb077b791606abdb186c4284956f03b58a629f2434a98b5e4bf8efcc32a7aa66791b16c57cf4da5cb83d9801e2d7db3d0e58e1ada401ddbc20c/download"
-- Fetching "https://itk.org/files/ExternalData/SHA512/0aad3941e82b5cb077b791606abdb186c4284956f03b58a629f2434a98b5e4bf8efcc32a7aa66791b16c57cf4da5cb83d9801e2d7db3d0e58e1ada401ddbc20c"
-- Fetching "https://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=0aad3941e82b5cb077b791606abdb186c4284956f03b58a629f2434a98b5e4bf8efcc32a7aa66791b16c57cf4da5cb83d9801e2d7db3d0e58e1ada401ddbc20c&algorithm=SHA512"
CMake Error at /usr/share/cmake/Modules/ExternalData.cmake:1121 (message):


  Object MD5=5970c467ac9b44e55829ccf0a1296fd1 not found at:

    https://insightsoftwareconsortium.github.io/ITKTestingData/MD5/5970c467ac9b44e55829ccf0a1296fd1 ("Couldn't resolve host name")
    https://data.kitware.com:443/api/v1/file/hashsum/MD5/5970c467ac9b44e55829ccf0a1296fd1/download ("Couldn't resolve host name")
    https://itk.org/files/ExternalData/MD5/5970c467ac9b44e55829ccf0a1296fd1 ("Couldn't resolve host name")
    https://slicer.kitware.com/midas3/api/rest?method=midas.bitstream.download&checksum=5970c467ac9b44e55829ccf0a1296fd1&algorithm=MD5 ("Couldn't resolve host name")

("Couldn't resolve host name")

Has that terminal from where you are building access to https?

The data is there, I tested it succesfully doing:

 curl -O https://insightsoftwareconsortium.github.io/ITKTestingData/SHA512/7978c22b34eea3afd202e0d29222272bb499b00e5444fccf60f929383eed7c15da1bc868db7f6d09e6dd30c4b9df477435a3bcc8c24fbd8bfcbdca8a4149711b 

[TheChymera]

Ah, I see, yes, the Gentoo installation sandbox is (and for good reason) rather adamant in restricting undocumented internet access at compile time. We explicitly keep track of all the checksums of all the files which need to be downloaded. I will have to see how I can download this explicitly outside of CMake.

What would I have to sed out of the cmake code to stop it from doing this dynamically? And what directory should I put the data into?

[dzenanz]

You can get the InsightData from ITK releases page, e.g. 4.13.2 or 5.0rc01.

Here is an explanation (might be out of date) how to update the external data from version to version. Just using the data should be accomplished by setting CMake configuration variable ExternalData_OBJECT_STORES to point to the directory where the data is. I guess that can be accomplished by setting the environment variable of that name to the wanted directory.

[phcerdan]

You can download the testing data from here.

And in your CMake options, pass the path to where the downloaded folder: .ExternalData is placed.

Example:

cmake ../ITK-src -DBUILD_TESTING:BOOL=ON -DExternalData_OBJECT_STORES:STRING=/path/Downloads/InsightToolkit-xxx/.ExternalData

[TheChymera]

@phcerdan @dzenanz

Thanks,

With the following cmake command:

cmake -C /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_common_config.cmake -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_SHARED_LIBS=ON -DITK_USE_SYSTEM_DCMTK=ON -DGDCM_USE_SYSTEM_OPENJPEG=ON -DITK_USE_SYSTEM_DOUBLECONVERSION=ON -DITK_USE_SYSTEM_CASTXML=ON -DITK_USE_SYSTEM_HDF5=ON -DITK_USE_SYSTEM_JPEG=ON -DITK_USE_SYSTEM_PNG=ON -DITK_USE_SYSTEM_SWIG=ON -DITK_USE_SYSTEM_TIFF=ON -DITK_USE_SYSTEM_ZLIB=ON -DITK_BUILD_DEFAULT_MODULES=ON -DITK_COMPUTER_MEMORY_SIZE=1 -DWRAP_ITK_JAVA=OFF -DWRAP_ITK_TCL=OFF -Ddouble-conversion_INCLUDE_DIRS=/usr/include/double-conversion -DBUILD_TESTING=ON -DBUILD_EXAMPLES=OFF -DExternalData_OBJECT_STORES=/var/tmp/portage/sci-libs/itk-5.0_rc1/work/InsightToolkit-5.0rc01/.ExternalData -DITK_USE_REVIEW=OFF -DITK_INSTALL_LIBRARY_DIR=lib64 -DITK_WRAP_PYTHON=OFF -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/gentoo_toolchain.cmake /var/tmp/portage/sci-libs/itk-5.0_rc1/work/ITK-5.0rc01

I get a new error log. IT sounds like the test suite actually fails, but I cannot locate the exact error:
build.log

[phcerdan]

Fails downloading KWStyle:

CMake Error at KWStyle-prefix/src/KWStyle-stamp/KWStyle-download-Gentoo.cmake:16 (message):
  Command failed: 1

   '/usr/bin/cmake' '-P' '/var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/KWStyle-prefix/tmp/KWStyle-gitclone.cmake'

  See also

    /var/tmp/portage/sci-libs/itk-5.0_rc1/work/itk-5.0_rc1_build/KWStyle-prefix/src/KWStyle-stamp/KWStyle-download-*.log

Try adding the CMake option:

-DITK_USE_KWSTYLE:BOOL=OFF

[TheChymera]

Thank you so much, this seems to have worked!