Mmdetection: Syntax error converting model to 2.0

Created on 25 Jun 2020  路  5Comments  路  Source: open-mmlab/mmdetection

  1. I have searched related issues but cannot get the expected help.
  2. using latest

Describe the bug
When running the upgrade_model_version I'm getting a syntax error:

Traceback (most recent call last):
  File "/Users/robertwashboourne/Documents/tables/env/lib/python3.7/site-packages/mmcv/utils/config.py", line 89, in _validate_py_syntax
    ast.parse(content)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    /content/drive/My Drive/chunk cascade_mask_rcnn_hrnetv2p_w32_20e.py
    ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tools/upgrade_model_version.py", line 193, in <module>
    main()
  File "tools/upgrade_model_version.py", line 189, in main
    convert(args.in_file, args.out_file, args.num_classes)
  File "tools/upgrade_model_version.py", line 126, in convert
    meta_info['config'])
  File "tools/upgrade_model_version.py", line 24, in parse_config
    config = Config.fromfile(config_path)
  File "/Users/robertwashboourne/Documents/tables/env/lib/python3.7/site-packages/mmcv/utils/config.py", line 178, in fromfile
    cfg_dict, cfg_text = Config._file2dict(filename)
  File "/Users/robertwashboourne/Documents/tables/env/lib/python3.7/site-packages/mmcv/utils/config.py", line 107, in _file2dict
    Config._validate_py_syntax(filename)
  File "/Users/robertwashboourne/Documents/tables/env/lib/python3.7/site-packages/mmcv/utils/config.py", line 91, in _validate_py_syntax
    raise SyntaxError('There are syntax errors in config '
SyntaxError: There are syntax errors in config file /var/folders/88/zdrrnbg14td2pf67pb4_2nf00000gn/T/tmpz98nf4lx.py
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x107a735f0>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tempfile.py", line 448, in __del__
    self.close()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tempfile.py", line 444, in close
    unlink(self.name)
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/88/zdrrnbg14td2pf67pb4_2nf00000gn/T/tmpltt4irkb/tmpbq5m84wy.py'

I ran a grep and "/content/drive/My Drive/chunk cascade_mask_rcnn_hrnetv2p_w32_20e.py" is in the pth file but I'm not sure why the script is failing. I'm on mmdetection master, latest pytorch

Environment

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
sys.platform: darwin
Python: 3.7.7 (default, Mar 10 2020, 15:43:33) [Clang 11.0.0 (clang-1100.0.33.17)]
CUDA available: False
GCC: Apple clang version 11.0.3 (clang-1103.0.32.62)
PyTorch: 1.5.1
PyTorch compiling details: PyTorch built with:
  - GCC 4.2
  - C++ Version: 201402
  - clang 9.1.0
  - Intel(R) Math Kernel Library Version 2019.0.5 Product Build 20190808 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_INTERNAL_THREADPOOL_IMPL -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unknown-warning-option -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-aligned-allocation-unavailable -Wno-c++14-extensions -Wno-constexpr-not-const -Wno-missing-braces -Qunused-arguments -fcolor-diagnostics -faligned-new -fno-math-errno -fno-trapping-math -Werror=format -Wno-unused-private-field -Wno-missing-braces -Wno-c++14-extensions -Wno-constexpr-not-const, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=OFF, USE_STATIC_DISPATCH=OFF, 

TorchVision: 0.6.1
OpenCV: 4.2.0
MMCV: 0.6.1
MMDetection: 2.1.0+184e793
MMDetection Compiler: clang 11.0.3
MMDetection CUDA Compiler: not available

This is the file I'm trying to convert: https://drive.google.com/file/d/1vfUr4pmmI4GICZubAnBeFU8pviMUW_c9/view

Most helpful comment

There is a syntax error in your config file. Please have a check.

I met the same situation, and I'm pretty sure it's a bug. The program writes the value of checkpoint['meta']['config'] in the Model file directly to the tmp.py file, but the first line is the path, so the first line of the generated py file is the path not commented out.
I solved the problem as following @hiyyg @rawsh-bt :

import torch
checkpoint = torch.load('model.pth')
checkpoint['meta']['config'] = '#' + checkpoint['meta']['config']
torch.save(checkpoint, 'new_model.pth')

All 5 comments

There is a syntax error in your config file. Please have a check.

@rawsh-bt I met similar issue when trying to upgrade old trained model to 2.x, how did you solve this error?

@hiyyg i did not solve it no. fixed syntax and still had errors

There is a syntax error in your config file. Please have a check.

I met the same situation, and I'm pretty sure it's a bug. The program writes the value of checkpoint['meta']['config'] in the Model file directly to the tmp.py file, but the first line is the path, so the first line of the generated py file is the path not commented out.
I solved the problem as following @hiyyg @rawsh-bt :

import torch
checkpoint = torch.load('model.pth')
checkpoint['meta']['config'] = '#' + checkpoint['meta']['config']
torch.save(checkpoint, 'new_model.pth')

@CescMessi that did it! thank you sm for posting the fix.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

Youngkl0726 picture Youngkl0726  路  3Comments

yangcong955 picture yangcong955  路  3Comments

tianxinhang picture tianxinhang  路  3Comments

hust-kevin picture hust-kevin  路  3Comments

songyuc picture songyuc  路  3Comments