Faster-rcnn.pytorch: A problem with sh make.sh

[WangTianYuan]

: not found make.sh:
: not found make.sh:
: not found: make.sh:
: not found: make.sh:
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

not recognized-inplace
make.sh: 15: make.sh: rm: not found
: not found: make.sh:
: not found: make.sh:
make.sh: 20: cd: can't cd to model/nms/src
Compiling nms kernels by nvcc...
nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified
make.sh: 23: make.sh: -D: not found
: not found: make.sh:
make.sh: 25: cd: can't cd to ../
': [Errno 2] No such file or directory
: not found: make.sh:
make.sh: 29: cd: can't cd to ../../
make.sh: 30: cd: can't cd to model/roi_pooling/src
Compiling roi pooling kernels by nvcc...
nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified
make.sh: 33: make.sh: -D: not found
make.sh: 34: cd: can't cd to ../
': [Errno 2] No such file or directory
: not found: make.sh:
make.sh: 38: cd: can't cd to ../../
make.sh: 39: cd: can't cd to model/roi_align/src
Compiling roi align kernels by nvcc...
nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified
make.sh: 42: make.sh: -D: not found
make.sh: 43: cd: can't cd to ../
': [Errno 2] No such file or directory
: not found: make.sh:
make.sh: 47: cd: can't cd to ../../
make.sh: 48: cd: can't cd to model/roi_crop/src
Compiling roi crop kernels by nvcc...
nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified
make.sh: 51: make.sh: -D: not found
make.sh: 52: cd: can't cd to ../
': [Errno 2] No such file or directory

So I do it step by step
python setup.py build_ext --inplace
rm -rf build

CUDA_ARCH="-gencode arch=compute_70,code=sm_70"

compile NMS

cd model/nms/src
echo "Compiling nms kernels by nvcc..."
nvcc -c -o nms_cuda_kernel.cu.o nms_cuda_kernel.cu
-D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC $CUDA_ARCH

The above is ok

cd ../
python build.py

this line I got the problem

Including CUDA code.
/home/wty/torch/faster-rcnn.pytorch/lib/model/nms
['/home/wty/torch/faster-rcnn.pytorch/lib/model/nms/src/nms_cuda_kernel.cu.o']
Traceback (most recent call last):
File "build.py", line 33, in
extra_objects=extra_objects
File "/usr/local/lib/python2.7/dist-packages/torch/utils/ffi/__init__.py", line 179, in create_extension
ffi.cdef(_typedefs + all_headers_source)
File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 107, in cdef
self._cdef(csource, override=override, packed=packed)
File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 121, in _cdef
self._parser.parse(csource, override=override, **options)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 315, in parse
self._internal_parse(csource)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 320, in _internal_parse
ast, macros, csource = self._parse(csource)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 278, in _parse
self.convert_pycparser_error(e, csource)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 307, in convert_pycparser_error
raise CDefError(msg)
cffi.error.CDefError: parse error
:31:1: Illegal character 'r'

I do the search and I don't know why it's wrong? could someone help me with this? thanks

[jwyang]

@WangTianYuan which version of cuda and pytorch are you using?

[WangTianYuan]

@jwyang cuda9.1+python2.7

[jwyang]

@WangTianYuan which version of pytorch? (not python)

[WangTianYuan]

@jwyang 0.4.0

[jwyang]

@WangTianYuan what's you exact command and which directory did you run it in?

[WangTianYuan]

cd model/nms/src
echo "Compiling nms kernels by nvcc..."
nvcc -c -o nms_cuda_kernel.cu.o nms_cuda_kernel.cu
-D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC $CUDA_ARCH
cd ../

python build.py
this is the exact command, I run it in model/nms

[jwyang]

@WangTianYuan what if you directly run sh make.sh in lib/ ?

[WangTianYuan]

: not found make.sh:
: not found make.sh:
: not found: make.sh:
: not found: make.sh:
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

You can see all log in my issue describe, I don't know why I can't run this .sh file

[jwyang]

make.sh is in your lib, but you cannot run it since it is not found, right?

[WangTianYuan]

yes.

[jwyang]

@WangTianYuan you run sh make.sh in lib folder right?

[WangTianYuan]

@jwyang yes, I first use ls to make sure it's the right folder

[HURB]

@WangTianYuan The code you download from github is not the UTF-8 encoded, you need to change the encoded before compiler it. copy to the Notepad and copy back is useful. I solved the problem in this way.

[WangTianYuan]

@HURB I've changed the make.sh to URF-8, It doesn't work, can you describe it more specific?

[HURB]

@WangTianYuan encoded all the python code

[WangTianYuan]

@HURB How? Do I need encode the file one by one or is there an easy way to do it?

[hongdayu]

@WangTianYuan The code you download from github is not the UTF-8 encoded, you need to change the encoded before compiler it. copy to the Notepad and copy back is useful. I solved the problem in this way.

tks

[WangTianYuan]

@HURB I still can't run this model, can you email the whole git.zip to me? my e-address is [email protected], thank you very much.

[WangTianYuan]

@HURB @hongdayu I'm sorry to interrupt you, but I follow your instruction:copy the file to notepad then copy back, I still have the same problem. Where did I do wrong?

[hongdayu]

@HURB @hongdayu I'm sorry to interrupt you, but I follow your instruction:copy the file to notepad then copy back, I still have the same problem. Where did I do wrong?
it works for me
create a new .sh file, copy the code in it.

[WangTianYuan]

@hongdayu I create new make.sh, build.py file, copy the code; then it still have the 'r' problem.

cffi.error.CDefError: parse error
<cdef source string>:31:1: Illegal character '\r'

Do I need to recreate all file?

[lwq1122]

maybe you download to windows, then you upload it to linux system?
you should change the file into unix fomat

1. vim filename
2. input : set ff, the format is dos, you should change it into unix (set ff=unix)
3. save it

[WangTianYuan]

@lwq1122 thx, I solved this problem by follow commands:
touch make.sh
vim make.sh
copy the entire text to notepad++, save as utf-8
then copy back

[huangbiubiu]

I think that's not the encoding problem. Files downloaded from Github is using CRLF, just change it to LF on Linux and that will work.

[ChengruZhu]

Try running dos2unix on the script.
dos2unix maks.sh