Mmdetection: Error in pip install -v -e .

Created on 4 Jun 2020 · 12Comments · Source: open-mmlab/mmdetection

Following is what I did:

conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab
conda install pytorch cudatoolkit=10.1 torchvision -c pytorch
git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection
pip install -r requirements/build.txt
pip install "git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI"
pip install -v -e .  # or "python setup.py develop"

The Error is:

 pip install -v -e .
Non-user install because site-packages writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-elygrjhk
Created temporary directory: /tmp/pip-req-tracker-xxvrgzg1
Initialized build tracking at /tmp/pip-req-tracker-xxvrgzg1
Created build tracker: /tmp/pip-req-tracker-xxvrgzg1
Entered build tracker: /tmp/pip-req-tracker-xxvrgzg1
Created temporary directory: /tmp/pip-install-pbsax23e
Obtaining file:///media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection
  Added file:///media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection to build tracker '/tmp/pip-req-tracker-xxvrgzg1'
    Running setup.py (path:/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py) egg_info for package from file:///media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection
    Running command python setup.py egg_info
    running egg_info
    writing mmdet.egg-info/PKG-INFO
    writing dependency_links to mmdet.egg-info/dependency_links.txt
    writing requirements to mmdet.egg-info/requires.txt
    writing top-level names to mmdet.egg-info/top_level.txt
    reading manifest file 'mmdet.egg-info/SOURCES.txt'
    writing manifest file 'mmdet.egg-info/SOURCES.txt'
  Source in /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection has version 2.0.0+e05e858, which satisfies requirement mmdet==2.0.0+e05e858 from file:///media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection
  Removed mmdet==2.0.0+e05e858 from file:///media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection from build tracker '/tmp/pip-req-tracker-xxvrgzg1'
Requirement already satisfied: matplotlib in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (3.2.1)
Requirement already satisfied: mmcv>=0.5.7 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (0.5.9)
Requirement already satisfied: numpy in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (1.18.1)
Requirement already satisfied: Pillow<=6.2.2 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (6.2.2)
Requirement already satisfied: six in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (1.14.0)
Requirement already satisfied: terminaltables in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (3.1.0)
Requirement already satisfied: torch>=1.3 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (1.5.0)
Requirement already satisfied: torchvision in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmdet==2.0.0+e05e858) (0.6.0a0+82fd1c8)
Requirement already satisfied: cycler>=0.10 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from matplotlib->mmdet==2.0.0+e05e858) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from matplotlib->mmdet==2.0.0+e05e858) (1.2.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from matplotlib->mmdet==2.0.0+e05e858) (2.4.7)
Requirement already satisfied: python-dateutil>=2.1 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from matplotlib->mmdet==2.0.0+e05e858) (2.8.1)
Requirement already satisfied: pyyaml in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmcv>=0.5.7->mmdet==2.0.0+e05e858) (5.3.1)
Requirement already satisfied: yapf in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmcv>=0.5.7->mmdet==2.0.0+e05e858) (0.30.0)
Requirement already satisfied: addict in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmcv>=0.5.7->mmdet==2.0.0+e05e858) (2.2.1)
Requirement already satisfied: opencv-python>=3 in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from mmcv>=0.5.7->mmdet==2.0.0+e05e858) (4.2.0.34)
Requirement already satisfied: future in /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages (from torch>=1.3->mmdet==2.0.0+e05e858) (0.18.2)
Installing collected packages: mmdet
  Running setup.py develop for mmdet
    Running command /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py'"'"'; __file__='"'"'/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
    running develop
    running egg_info
    writing mmdet.egg-info/PKG-INFO
    writing dependency_links to mmdet.egg-info/dependency_links.txt
    writing requirements to mmdet.egg-info/requires.txt
    writing top-level names to mmdet.egg-info/top_level.txt
    reading manifest file 'mmdet.egg-info/SOURCES.txt'
    writing manifest file 'mmdet.egg-info/SOURCES.txt'
    running build_ext
    building 'mmdet.ops.utils.compiling_info' extension
    Emitting ninja build file /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/build/temp.linux-x86_64-3.7/build.ninja...
    Compiling objects...
    Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
    [1/1] c++ -MMD -MF /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/build/temp.linux-x86_64-3.7/mmdet/ops/utils/src/compiling_info.o.d -pthread -B /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include/TH -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/include/python3.7m -c -c /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/mmdet/ops/utils/src/compiling_info.cpp -o /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/build/temp.linux-x86_64-3.7/mmdet/ops/utils/src/compiling_info.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=compiling_info -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
    FAILED: /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/build/temp.linux-x86_64-3.7/mmdet/ops/utils/src/compiling_info.o
    c++ -MMD -MF /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/build/temp.linux-x86_64-3.7/mmdet/ops/utils/src/compiling_info.o.d -pthread -B /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include/TH -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/include/python3.7m -c -c /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/mmdet/ops/utils/src/compiling_info.cpp -o /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/build/temp.linux-x86_64-3.7/mmdet/ops/utils/src/compiling_info.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=compiling_info -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    /media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/mmdet/ops/utils/src/compiling_info.cpp:6:30: fatal error: cuda_runtime_api.h: No such file or directory
    compilation terminated.
    ninja: build stopped: subcommand failed.
    Traceback (most recent call last):
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1400, in _run_ninja_build
        check=True)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/subprocess.py", line 512, in run
        output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py", line 300, in <module>
        zip_safe=False)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/setuptools/__init__.py", line 161, in setup
        return distutils.core.setup(**attrs)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
        _build_ext.run(self)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
        _build_ext.build_ext.run(self)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
        build_ext.build_extensions(self)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
        _build_ext.build_ext.build_extensions(self)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
        _build_ext.build_extension(self, ext)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
        depends=ext.depends)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 423, in unix_wrap_ninja_compile
        with_cuda=with_cuda)
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects
        error_prefix='Error compiling objects for extension')
      File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
        raise RuntimeError(message)
    RuntimeError: Error compiling objects for extension
Cleaning up...
Removed build tracker: '/tmp/pip-req-tracker-xxvrgzg1'
ERROR: Command errored out with exit status 1: /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py'"'"'; __file__='"'"'/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
Exception information:
Traceback (most recent call last):
  File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 404, in run
    use_user_site=options.use_user_site,
  File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/req/__init__.py", line 71, in install_given_reqs
    **kwargs
  File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 802, in install
    unpacked_source_directory=self.unpacked_source_directory,
  File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/operations/install/editable_legacy.py", line 51, in install_editable
    cwd=unpacked_source_directory,
  File "/media/Zeus/Segmentation/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/utils/subprocess.py", line 242, in call_subprocess
    raise InstallationError(exc_msg)
pip._internal.exceptions.InstallationError: Command errored out with exit status 1: /media/Zeus/Segmentation/anaconda3/envs/open-mmlab/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py'"'"'; __file__='"'"'/media/Zeus/Segmentation/shelf_detection_xzf/test/mmdetection/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

This is the output from nvidia-smi

| NVIDIA-SMI 418.39       Driver Version: 418.39       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  TITAN RTX           Off  | 00000000:04:00.0 Off |                  N/A |
| 41%   41C    P2    95W / 280W |   1985MiB / 24190MiB |     10%      Default |
+-------------------------------+----------------------+----------------------+
|   1  TITAN RTX           Off  | 00000000:05:00.0 Off |                  N/A |
| 41%   57C    P2    88W / 280W |    935MiB / 24190MiB |     15%      Default |
+-------------------------------+----------------------+----------------------+
|   2  TITAN RTX           Off  | 00000000:08:00.0 Off |                  N/A |
| 50%   70C    P2   115W / 280W |    935MiB / 24190MiB |      5%      Default |
+-------------------------------+----------------------+----------------------+
|   3  TITAN RTX           Off  | 00000000:09:00.0 Off |                  N/A |
| 41%   61C    P2   101W / 280W |   3427MiB / 24190MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   4  TITAN RTX           Off  | 00000000:84:00.0 Off |                  N/A |
| 40%   58C    P2    96W / 280W |   7405MiB / 24190MiB |      1%      Default |
+-------------------------------+----------------------+----------------------+
|   5  TITAN RTX           Off  | 00000000:85:00.0 Off |                  N/A |
| 40%   55C    P2    86W / 280W |   7085MiB / 24190MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   6  TITAN RTX           Off  | 00000000:88:00.0 Off |                  N/A |
| 40%   45C    P2    88W / 280W |   6531MiB / 24190MiB |      3%      Default |
+-------------------------------+----------------------+----------------------+
|   7  TITAN RTX           Off  | 00000000:89:00.0 Off |                  N/A |
| 40%   37C    P0    50W / 280W |      0MiB / 24190MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     11989      C   python3                                     1967MiB |
|    1     11989      C   python3                                      925MiB |
|    2     11989      C   python3                                      925MiB |
|    3     11989      C   python3                                      925MiB |
|    4     40038      C   python3                                     7395MiB |
|    5     40038      C   python3                                     7075MiB |
|    6     40038      C   python3                                     6521MiB |
+-----------------------------------------------------------------------------+

Source

Zhefan-Xu

Most helpful comment

I tried both cuda 10.1 and 10.2 on ubuntu 18.04, and no error occurs.

That's weird...

I figure this out by export CUDA_HOME=/usr/local/cuda-10.1.

Zhefan-Xu on 8 Jun 2020

👍5 ❤1

All 12 comments

Could you provide more environment info, e.g., OS, GCC version, etc.

hellock on 7 Jun 2020

Could you provide more environment info, e.g., OS, GCC version, etc.

Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-64-generic x86_64)
gcc 5.4.0

Please let me know if you need more info!!! THanks!!!!

Zhefan-Xu on 8 Jun 2020

Could you provide more environment info, e.g., OS, GCC version, etc.

If I install torch cudatoolkit=10.2, there is no problem when executing pip install -v -e .
But my cuda version is 10.1 and I cannot upgrade it since it is not a personal machine.

Zhefan-Xu on 8 Jun 2020

I encouter the same problem.
My enviroment is cuda10.2, pytorch 1.5,torchvision 0.6,gcc 7.5,python 3.7.6 in ubuntu18.04.