Onnxruntime: building with --cmake_extra_defines doesn't work properly with multiple items

Created on 6 Dec 2019  路  6Comments  路  Source: microsoft/onnxruntime

Describe the bug
If you run the build with multple items in the --cmake_extra_defines argument it is not handled properly. Instead of getting multiple args, all the args are in one string.

Urgency
You can work around this by running running .build.sh mutlple times without clearing the CMakeCache.txt for by modifying the CMakeCache.txt by hand.

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS 7
  • ONNX Runtime installed from (source or binary): git clone
  • ONNX Runtime version: 1.0.0
  • Python version: 3.6.8
  • Visual Studio version (if applicable):
  • GCC/Compiler version (if compiling from source): 9.2.0
  • CUDA/cuDNN version: n/a
  • GPU model and memory: n/a

To Reproduce
Describe steps/code to reproduce the behavior:
Run something like

./build.sh --cmake_extra_defines "PYTHON_LIBRARY=/somewhere CMAKE_CXX_FLAGS=-Wflaghere"

Expected behavior
Should produce arguments like

"-DPYTHON_LIBRARY=/somewhere", "-DCMAKE_CXX_FLAGS=-Wflaghere"

Instead, it produces this

"-DPYTHON_LIBRARY=/somewhere CMAKE_CXX_FLAGS=-Wflaghere

Which leads to only the first argument being set in the CMakeCache.txt, but it set to

/somewhere CMAKE_CXX_FLAGS=-Wflaghere

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.

I think this issue is at https://github.com/microsoft/onnxruntime/blob/v1.0.0/tools/ci_build/build.py#L410

the variable cmake_extra_defines is a list of 1 string, ["PYTHON_LIBRARY=/somewhere CMAKE_CXX_FLAGS=-Wflaghere"] in this example

Changing the line to the following fixed my issue.

```
cmake_args += ["-D{}".format(d) for define in cmake_extra_defines for d in define.split(" ")]
````

It is not very robust, ie doesn't handle multiple spaces etc, but it worked for my needs.

contributions-welcome enhancement

Most helpful comment

I don't believe you need to quote the multiple items. The following works for me to set two different cmake defines

--cmake_extra_defines CMAKE_CXX_COMPILER=/usr/bin/g++-4.8 CMAKE_C_COMPILER=/usr/bin/gcc-4.8

All 6 comments

If you know how to fix it, welcome to contribute your change.

Thanks for the feedback @snnn. I thought about contributing but have no idea how to write a test for this. I am happy to make a PR though

You can do it without adding unitests.

@snnn I found a couple of places where it looks like cmake_extra_defines are using more than one variable. For example azure-pipelines-py-packaging.yml and linux-ort-srv-nightly-pipeline.yml

The first one looks like it is using build.py. The second is being called from a script, but I didn't see any code that looks like it munging parameters.

Is there somewhere I can look further to see if the cmake_extra_defines arg is broken for those usages?

Thanks

I don't believe you need to quote the multiple items. The following works for me to set two different cmake defines

--cmake_extra_defines CMAKE_CXX_COMPILER=/usr/bin/g++-4.8 CMAKE_C_COMPILER=/usr/bin/gcc-4.8

Was this page helpful?
0 / 5 - 0 ratings