Detectron2: Build Error on Mac OS

[uestcyile]

When I use the common "CC=clang CXX=clang++ python -m pip install -e ." to build detectron2 from source, the following error occurred:

Instructions To Reproduce the Issue:

1. what changes you made (git diff) or what code you wrote

1. what exact command you run:

CC=clang CXX=clang++ python -m pip install -e .

1. what you observed (including the full logs):

ERROR: Command errored out with exit status 1:
     command: /Users/user/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/user/Documents/Code/detectron2/setup.py'"'"'; __file__='"'"'/Users/user/Documents/Code/detectron2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/user/Documents/Code/detectron2/
    Complete output (156 lines):
    running develop
    running egg_info
    writing detectron2.egg-info/PKG-INFO
    writing dependency_links to detectron2.egg-info/dependency_links.txt
    writing requirements to detectron2.egg-info/requires.txt
    writing top-level names to detectron2.egg-info/top_level.txt
    reading manifest file 'detectron2.egg-info/SOURCES.txt'
    writing manifest file 'detectron2.egg-info/SOURCES.txt'
    running build_ext
    building 'detectron2._C' extension
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/user/anaconda3/include -arch x86_64 -I/Users/user/anaconda3/include -arch x86_64 -I/Users/user/Documents/Code/detectron2/detectron2/layers/csrc -I/Users/user/anaconda3/lib/python3.7/site-packages/torch/include -I/Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/Users/user/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/Users/user/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/Users/user/anaconda3/include/python3.7m -c /Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.cpp -o build/temp.macosx-10.7-x86_64-3.7/Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
    In file included from /Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.cpp:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:3:
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/DeviceType.h:65:20: error: explicit specialization of non-template struct 'hash'
    template <> struct hash<c10::DeviceType> {
                       ^   ~~~~~~~~~~~~~~~~~
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/DeviceType.h:67:21: error: expected '(' for function-style cast or type construction
        return std::hash<int>()(static_cast<int>(k));
               ~~~~~~~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/DeviceType.h:67:25: error: expected '(' for function-style cast or type construction
        return std::hash<int>()(static_cast<int>(k));
                         ~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/DeviceType.h:67:27: error: expected expression
        return std::hash<int>()(static_cast<int>(k));
                              ^
    In file included from /Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.cpp:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:5:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/StringUtil.h:5:
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/string_utils.h:68:12: error: no member named 'stod' in namespace 'std'
    using std::stod;
          ~~~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/string_utils.h:69:12: error: no member named 'stoi' in namespace 'std'; did you mean 'atoi'?
    using std::stoi;
          ~~~~~^~~~
               atoi
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/stdlib.h:132:6: note: 'atoi' declared here
    int      atoi(const char *);
             ^
    In file included from /Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.cpp:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:5:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/StringUtil.h:5:
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/string_utils.h:70:12: error: no member named 'stoull' in namespace 'std'
    using std::stoull;
          ~~~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/string_utils.h:71:12: error: no member named 'stoll' in namespace 'std'; did you mean 'atoll'?
    using std::stoll;
          ~~~~~^~~~~
               atoll
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/stdlib.h:136:3: note: 'atoll' declared here
             atoll(const char *);
             ^
    In file included from /Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.cpp:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:5:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/StringUtil.h:5:
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/string_utils.h:72:12: error: no member named 'to_string' in namespace 'std'
    using std::to_string;
          ~~~~~^
    In file included from /Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.cpp:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6:
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:111:8: error: explicit specialization of non-template struct 'hash'
    struct hash<c10::Device> {
           ^   ~~~~~~~~~~~~~
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:111:8: error: redefinition of 'hash'
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/DeviceType.h:65:20: note: previous definition is here
    template <> struct hash<c10::DeviceType> {
                       ^
    In file included from /Users/user/Documents/Code/detectron2/detectron2/layers/csrc/vision.cpp:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:3:
    In file included from /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:7:
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:42:8: error: no type named 'unique_ptr' in namespace 'std'
      std::unique_ptr<void, DeleterFnPtr> ctx_;
      ~~~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:42:18: error: expected member name or ';' after declaration specifiers
      std::unique_ptr<void, DeleterFnPtr> ctx_;
      ~~~~~~~~~~~~~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:66:8: error: no type named 'unique_ptr' in namespace 'std'
      std::unique_ptr<void, DeleterFnPtr>&& move_context() {
      ~~~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:66:18: error: expected member name or ';' after declaration specifiers
      std::unique_ptr<void, DeleterFnPtr>&& move_context() {
      ~~~~~~~~~~~~~~~^
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:45:37: error: member initializer 'ctx_' does not name a non-static data member or base class
      UniqueVoidPtr() : data_(nullptr), ctx_(nullptr, &deleteNothing) {}
                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:47:22: error: member initializer 'ctx_' does not name a non-static data member or base class
          : data_(data), ctx_(nullptr, &deleteNothing) {}
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:49:22: error: member initializer 'ctx_' does not name a non-static data member or base class
          : data_(data), ctx_(ctx, ctx_deleter ? ctx_deleter : &deleteNothing) {}
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /Users/user/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/UniqueVoidPtr.h:54:5: error: use of undeclared identifier 'ctx_'
        ctx_ = nullptr;
        ^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 errors generated.
    error: command 'clang' failed with exit status 1
    ----------------------------------------
  ERROR: Can't roll back detectron2; was not uninstalled
ERROR: Command errored out with exit status 1: /Users/user/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/user/Documents/Code/detectron2/setup.py'"'"'; __file__='"'"'/Users/user/Documents/Code/detectron2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Environment:

After run python -m detectron2.utils.collect_env in the environment, the output are:

---

sys.platform darwin
Python 3.7.3 (default, Mar 27 2019, 16:54:48) [Clang 4.0.1 (tags/RELEASE_401/final)]
numpy 1.16.2
detectron2._C failed to import
DETECTRON2_ENV_MODULE
PyTorch 1.4.0 @/Users/user/anaconda3/lib/python3.7/site-packages/torch
PyTorch debug build False
CUDA available False
Pillow 5.4.1
torchvision 0.5.0 @/Users/user/anaconda3/lib/python3.7/site-packages/torchvision
cv2 4.2.0

---

PyTorch built with:

• GCC 4.2
• clang 9.0.0
• Intel(R) Math Kernel Library Version 2019.0.5 Product Build 20190808 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
• NNPACK is enabled
• Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unknown-warning-option -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-aligned-allocation-unavailable -Wno-c++14-extensions -Wno-constexpr-not-const -Wno-missing-braces -Qunused-arguments -fcolor-diagnostics -faligned-new -fno-math-errno -fno-trapping-math -Wno-unused-private-field -Wno-missing-braces -Wno-c++14-extensions -Wno-constexpr-not-const, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=OFF, USE_STATIC_DISPATCH=OFF,

[ppwwyyxx]

Can you try MACOSX_DEPLOYMENT_TARGET=10.9 before the command?
If this doesn't work you probably need to upgrade xcode.

[uestcyile]

Can you try MACOSX_DEPLOYMENT_TARGET=10.9 before the command?
If this doesn't work you probably need to upgrade xcode.

After run export MACOSX_DEPLOYMENT_TARGET=10.9, then rebuild with CC=clang CXX=clang++ python -m pip install -e ., still something wrong as follows:

error: thread-local storage is not supported for the current target
      static thread_local ArgumentStash stash;
fatal error: recursive template instantiation exceeded maximum depth of 256
    template<size_t N, size_t ...S> struct make_index_sequence_impl : make_index_sequence_impl <N - 1, N - 1, S...> { };

dose it mean that I must upgrade my Xcode?

[ppwwyyxx]

According to google it does look like you need newer version of xcode.
https://stackoverflow.com/questions/33358417/how-to-get-support-for-thread-local-on-mac-osx-clang

[uestcyile]

According to google it does look like you need newer version of xcode.
https://stackoverflow.com/questions/33358417/how-to-get-support-for-thread-local-on-mac-osx-clang

Ok, I will upgrade my Xcode later, and then try to build again.
Thanks very much.