Yolact: convert yolact to ONNX

Created on 23 Jun 2019 · 57Comments · Source: dbolya/yolact

Hello again,
I'm try to convert yolact to ONNX with the following code:

weights_path = '/home/ws/DL/yolact/weights/yolact_im700_54_800000.pth'

import torch
import torch.onnx
import yolact
import torchvision

model = yolact.Yolact()

# state_dict = torch.load(weights_path)
# model.load_state_dict(state_dict)

model.load_weights(weights_path)

dummy_input = torch.randn(1, 3, 640, 480)

torch.onnx.export(model, dummy_input, "onnx_model_name.onnx")

error msg:

/home/ws/DL/yolact/yolact.py:256: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  for j, i in product(range(conv_h), range(conv_w)):
/home/ws/DL/yolact/yolact.py:279: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  self.priors = torch.Tensor(prior_data).view(-1, 4)
/home/ws/DL/yolact/yolact.py:279: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  self.priors = torch.Tensor(prior_data).view(-1, 4)
/home/ws/DL/yolact/layers/functions/detection.py:74: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  for batch_idx in range(batch_size):
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-a796dc0eef97> in <module>
     13 dummy_input = torch.randn(1, 3, 700, 700)
     14 
---> 15 torch.onnx.export(model, dummy_input, "onnx_model_name.onnx")

~/.local/lib/python3.6/site-packages/torch/onnx/__init__.py in export(*args, **kwargs)
     23 def export(*args, **kwargs):
     24     from torch.onnx import utils
---> 25     return utils.export(*args, **kwargs)
     26 
     27 

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in export(model, args, f, export_params, verbose, training, input_names, output_names, aten, export_raw_ir, operator_export_type, opset_version, _retain_param_name, do_constant_folding, strip_doc_string)
    129             operator_export_type=operator_export_type, opset_version=opset_version,
    130             _retain_param_name=_retain_param_name, do_constant_folding=do_constant_folding,
--> 131             strip_doc_string=strip_doc_string)
    132 
    133 

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in _export(model, args, f, export_params, verbose, training, input_names, output_names, operator_export_type, export_type, example_outputs, propagate, opset_version, _retain_param_name, do_constant_folding, strip_doc_string)
    361                                                         output_names, operator_export_type,
    362                                                         example_outputs, propagate,
--> 363                                                         _retain_param_name, do_constant_folding)
    364 
    365         # TODO: Don't allocate a in-memory string for the protobuf

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in _model_to_graph(model, args, verbose, training, input_names, output_names, operator_export_type, example_outputs, propagate, _retain_param_name, do_constant_folding, _disable_torch_constant_prop)
    264             model.graph, tuple(args), example_outputs, False, propagate)
    265     else:
--> 266         graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
    267         state_dict = _unique_state_dict(model)
    268         params = list(state_dict.values())

~/.local/lib/python3.6/site-packages/torch/onnx/utils.py in _trace_and_get_graph_from_model(model, args, training)
    223     # training mode was.)
    224     with set_training(model, training):
--> 225         trace, torch_out = torch.jit.get_trace_graph(model, args, _force_outplace=True)
    226 
    227     if orig_state_dict_keys != _unique_state_dict(model).keys():

~/.local/lib/python3.6/site-packages/torch/jit/__init__.py in get_trace_graph(f, args, kwargs, _force_outplace, return_inputs)
    229     if not isinstance(args, tuple):
    230         args = (args,)
--> 231     return LegacyTracedModule(f, _force_outplace, return_inputs)(*args, **kwargs)
    232 
    233 

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/.local/lib/python3.6/site-packages/torch/jit/__init__.py in forward(self, *args)
    292         try:
    293             trace_inputs = _unflatten(all_trace_inputs[:len(in_vars)], in_desc)
--> 294             out = self.inner(*trace_inputs)
    295             out_vars, _ = _flatten(out)
    296             torch._C._tracer_exit(tuple(out_vars))

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    489             hook(self, input)
    490         if torch._C._get_tracing_state():
--> 491             result = self._slow_forward(*input, **kwargs)
    492         else:
    493             result = self.forward(*input, **kwargs)

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
    479         tracing_state._traced_module_stack.append(self)
    480         try:
--> 481             result = self.forward(*input, **kwargs)
    482         finally:
    483             tracing_state.pop_scope()

~/DL/yolact/yolact.py in forward(self, x)
    615                 pred_outs['conf'] = F.softmax(pred_outs['conf'], -1)
    616 
--> 617             return self.detect(pred_outs)
    618 
    619 

~/DL/yolact/layers/functions/detection.py in __call__(self, predictions)
     73 
     74             for batch_idx in range(batch_size):
---> 75                 decoded_boxes = decode(loc_data[batch_idx], prior_data)
     76                 result = self.detect(batch_idx, conf_preds, decoded_boxes, mask_data, inst_data)
     77 

RuntimeError: isTensor() ASSERT FAILED at /pytorch/aten/src/ATen/core/ivalue.h:209, please report a bug to PyTorch. (toTensor at /pytorch/aten/src/ATen/core/ivalue.h:209)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7f721e0ac441 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7f721e0abd7a in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #2: <unknown function> + 0x979ad2 (0x7f721d130ad2 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #3: torch::jit::tracer::getNestedValueTrace(c10::IValue const&) + 0x41 (0x7f721d3939a1 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #4: <unknown function> + 0xa7651b (0x7f721d22d51b in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #5: <unknown function> + 0xa766db (0x7f721d22d6db in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #6: <unknown function> + 0x457942 (0x7f725d6d2942 in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #7: <unknown function> + 0x130cfc (0x7f725d3abcfc in /home/ws/.local/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #8: _PyCFunction_FastCallDict + 0x35c (0x56204c in /usr/bin/python3)
frame #9: /usr/bin/python3() [0x5a1501]
frame #10: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #11: /usr/bin/python3() [0x5136c6]
frame #12: _PyObject_FastCallKeywords + 0x19c (0x57ec0c in /usr/bin/python3)
frame #13: /usr/bin/python3() [0x4f88ba]
frame #14: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #15: _PyFunction_FastCallDict + 0xf5 (0x4f4065 in /usr/bin/python3)
frame #16: /usr/bin/python3() [0x5a1481]
frame #17: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #18: /usr/bin/python3() [0x513601]
frame #19: _PyObject_FastCallKeywords + 0x19c (0x57ec0c in /usr/bin/python3)
frame #20: /usr/bin/python3() [0x4f88ba]
frame #21: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #22: /usr/bin/python3() [0x4f6128]
frame #23: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #24: /usr/bin/python3() [0x5a1481]
frame #25: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #26: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #27: /usr/bin/python3() [0x4f6128]
frame #28: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #29: /usr/bin/python3() [0x5a1481]
frame #30: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #31: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #32: /usr/bin/python3() [0x4f6128]
frame #33: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #34: /usr/bin/python3() [0x5a1481]
frame #35: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #36: /usr/bin/python3() [0x513601]
frame #37: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #38: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #39: /usr/bin/python3() [0x4f6128]
frame #40: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #41: /usr/bin/python3() [0x5a1481]
frame #42: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #43: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #44: /usr/bin/python3() [0x4f6128]
frame #45: _PyFunction_FastCallDict + 0x2fe (0x4f426e in /usr/bin/python3)
frame #46: /usr/bin/python3() [0x5a1481]
frame #47: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #48: /usr/bin/python3() [0x513601]
frame #49: PyObject_Call + 0x3e (0x57c2fe in /usr/bin/python3)
frame #50: _PyEval_EvalFrameDefault + 0x1851 (0x4facb1 in /usr/bin/python3)
frame #51: /usr/bin/python3() [0x4f6128]
frame #52: /usr/bin/python3() [0x4f7d60]
frame #53: /usr/bin/python3() [0x4f876d]
frame #54: _PyEval_EvalFrameDefault + 0x1260 (0x4fa6c0 in /usr/bin/python3)
frame #55: /usr/bin/python3() [0x4f7a28]
frame #56: /usr/bin/python3() [0x4f876d]
frame #57: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #58: /usr/bin/python3() [0x4f6128]
frame #59: /usr/bin/python3() [0x4f7d60]
frame #60: /usr/bin/python3() [0x4f876d]
frame #61: _PyEval_EvalFrameDefault + 0x467 (0x4f98c7 in /usr/bin/python3)
frame #62: /usr/bin/python3() [0x4f6128]
frame #63: /usr/bin/python3() [0x4f7d60]

Source

sdimantsd

Most helpful comment

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1

Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

Ma-Dan on 22 Jul 2019

👍13 ❤4 🚀1 🎉1

All 57 comments

See #59. You'll have to put some elbow grease in if you want to get YOLACT traceable (i.e., exportable to ONNX) since I use a lot of pythonic code. I hear @Wilber529 was able to do it following these steps: https://github.com/dbolya/yolact/issues/59#issuecomment-501609792. You have to rewrite how I pass around variables (dictionaries are not supported I think) and you'll have to rewrite anything after Yolact's forward function (starting with self.detect) in your target language because I wrote it in a super pythonic way to make the model faster.

dbolya on 23 Jun 2019

Hi @dbolya thanks for you'r answer!
I'm not sure I understood you, Can you please expand?

sdimantsd on 23 Jun 2019

Yolact does not support conversion to ONNX, which is why you get an error. You'd need to change a lot of things to get conversion to ONNX to work, as outlined by @Wilber529 in https://github.com/dbolya/yolact/issues/59#issuecomment-501609792. I'm not making these changes to the main branch because they'd make the Python version run slower and make it harder to develop.

dbolya on 23 Jun 2019

thx

sdimantsd on 2 Jul 2019

I have converted yolact to onnx without Detect part, and also modified some upsampling code.
https://github.com/Ma-Dan/yolact/tree/onnx
Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods.
I also converted onnx model to CoreML model, 4 custom layers need to be implemented to make it work.

Ma-Dan on 12 Jul 2019

@Ma-Dan thanks for sharing the reference code ,i shalll look into this process and get back to if i have queries

abhigoku10 on 12 Jul 2019

@Ma-Dan thank you very much for sharing your work.
I am wondering, what needs to be implemented to execute the onnx model again.
What does this mean? "Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods."
Thanks for your help!

aweissen1 on 18 Jul 2019

@Ma-Dan Thank you for your code! I convert the model to onnx ,but the results is different from pytorch outpus,such loc , mask and proto, but conf is same! Do you see the problem?

ABlueLight on 18 Jul 2019

@abhigoku10 actually I just used the onnx branch from Ma-Dan to create an onnx file. Do you get an error while converting?

aweissen1 on 18 Jul 2019

@aweissen1 i was facing some package issues i shall look into to more in depth and solve it , where there any difference i the output generated

abhigoku10 on 18 Jul 2019

@Ma-Dan Hi, i convert to onnx ssuccessfully，but i found results is not corrent . can you share the version of the pytorch and onnxruntime are you using? Thx

ABlueLight on 19 Jul 2019

@Ma-Dan Can you give more information about the package dependencies for your Yolact-ONNX implementation?
And also, have compared the results of Yolact and that of your Yolact-ONNX implementations? If so, please give us insight on it.

sicarioakki on 22 Jul 2019

@ABlueLight and @aweissen1 should us the base code given by @Ma-Dan and train the model , or just load the trained model with this code what is the command to be used . Please share the process
Can i run it on gpu how much fps r u getting

abhigoku10 on 22 Jul 2019

i convert to onnx successfully and results is correct, today. Thx @Ma-Dan
@sicarioakki @abhigoku10 My package dependencies include pytorch1.0.0 torchvision0.2.1 onnx-tf1.3.0 onnxruntime0.4.0 onnx1.5.0 tensorflow-gpu1.14.0.
Just use @Ma-Dan code is ok , i don't modify the codes,just replaced my trained model.
Mayby the package version is a Important factors.

ABlueLight on 22 Jul 2019

@ABlueLight after conversion to onnx which platform are you going to deploy it . and did u convert to tensorflow based model

abhigoku10 on 22 Jul 2019

@abhigoku10 TensorFlow and it can run correctly

ABlueLight on 22 Jul 2019

Sorry for the delayed reply, I just fixed code on my repo to use correct onnx output.
https://github.com/Ma-Dan/yolact/commit/a0648974369762445bc2095c2318f3f5c7fb7297
The previous version move prior constant output to a separate file to make CoreML file correct, and I forgot to fix onnx output index. Sorry again!
And also notice that to make conversion to onnx correct, I hard coded sizes here.
https://github.com/Ma-Dan/yolact/blob/onnx/yolact.py#L344
So this code could not work correctly on yolact_im700_54_800000.pth weight, you need to fix the size here.

Ma-Dan on 22 Jul 2019

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1

Ma-Dan on 22 Jul 2019

👍13 ❤4 🚀1 🎉1

@Ma-Dan thanks for the response, i have few queries

the onnx model which you obtained can be used with C++
can i convert that model to other framework like tf or caffe
In the command shared u have mentioned as --cuda=False , does it mean that it can run only on CPU and not on GPU , i wanted to run it on GPU

abhigoku10 on 22 Jul 2019

@Ma-Dan Thank you! Great job.

aweissen1 on 22 Jul 2019

@ABlueLight how did you import it to Tensorflow?

ridasalam on 22 Jul 2019

file_name= yolact_base_0_4000.onnx
params= ['yolact', 'base', '0', '4000']
model_name= yolact_base
epoch= 0
iteration= 4000
Config not specified. Parsed yolact_base_config from the file name.

Loading model...Traceback (most recent call last):
File "onnxeval.py", line 1035, in
net.load_weights(args.trained_model)
File "/home/aeye/yolact-onnx/yolact_onnx_1/yolact.py", line 469, in load_weights
state_dict = torch.load(path, map_location='cpu')
File "/home/aeye/yolact-onnx/Yolact_ONNX/lib/python3.6/site-packages/torch/serialization.py", line 368, in load
return _load(f, map_location, pickle_module)
File "/home/aeye/yolact-onnx/Yolact_ONNX/lib/python3.6/site-packages/torch/serialization.py", line 532, in _load
magic_number = pickle_module.load(f)
_pickle.UnpicklingError: invalid load key, '\x08'.

I was able to covert the model to .onnx format.
But while inferencing, i am facing the above issue.

sicarioakki on 23 Jul 2019

@ABlueLight how did you import it to Tensorflow?
https://github.com/onnx/onnx-tensorflow

ABlueLight on 23 Jul 2019

@aweissen1 @ABlueLight hi guys , i am facing the same issue as above in my inference after conversion

file_name= yolact_base_0_4000.onnx
params= ['yolact', 'base', '0', '4000']
model_name= yolact_base
epoch= 0
iteration= 4000
Config not specified. Parsed yolact_base_config from the file name.

Any suggestions

abhigoku10 on 23 Jul 2019

@Ma-Dan @aweissen1 @ABlueLight How are guys able to load the ONNX model using torch.load() function? Only onnx.load() can be used right?

sicarioakki on 23 Jul 2019

@ABlueLight, do you have a huge difference in speed of inference?

I used @Ma-Dan 's helpful work to generate yolact.onnx and I load it through onnx load and onnx_tf.backend import prepare. All other post processing is still torch based. It takes 2 mins per image inference (compared to a couple of seconds in Pytorch)

Also, were you able to convert it to pure Tensorflow? (use Tensorflow pb file instead of onnx)

ridasalam on 23 Jul 2019

@ridasalam I convert it to pure tensorflow and it const about 400~500ms on i5 cpu。
On GPU，pytorch and tensorflow cost time are almost equal.

ABlueLight on 30 Jul 2019

@sicarioakki ONNX model should be loaded by onnx.load(),i think..

ABlueLight on 30 Jul 2019

@ridasalam I convert it to pure tensorflow and it const about 400~500ms on i5 cpu。
On GPU，pytorch and tensorflow cost time are almost equal.

Can you share the project of tensorflow?

sdimantsd on 14 Nov 2019

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1

Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

Hi, thanks for your codes, but I did not find the codes to generate onnx in eval.py, could anybody share the link?

JINGTING92 on 19 Nov 2019

Can someone provide their method for converting YOLACT to ONNX? I'm not getting sufficient speed on a Jetson Xavier, and would like to try C++ solution using TensorRT.

NinjaPerson24119 on 19 Nov 2019

Can someone provide their method for converting YOLACT to ONNX? I'm not getting sufficient speed on a Jetson Xavier, and would like to try C++ solution using TensorRT.

How fast was it running when running with Xavier?
Sorry for the issue

akaneko1019 on 26 Nov 2019

~3Hz with fp16 using darknet backend at 550x550

On Mon, Nov 25, 2019, 6:41 PM akaneko1019, notifications@github.com wrote:

Can someone provide their method for converting YOLACT to ONNX? I'm not
getting sufficient speed on a Jetson Xavier, and would like to try C++
solution using TensorRT.

How fast was it running when running with Xavier?
Sorry for the issue

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dbolya/yolact/issues/74?email_source=notifications&email_token=AHXXQOUBFADHCZRNXIAA3L3QVR5FHA5CNFSM4H2YZ7SKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFEM7HI#issuecomment-558419869,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHXXQOULURAICZFC7YPT22LQVR5FHANCNFSM4H2YZ7SA
.

NinjaPerson24119 on 26 Nov 2019

@JING switch to branch onnx. =)

HoangTienDuc on 28 Feb 2020

👍1

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1

Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned.
the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/__init__.py:134: UserWarning:
Found GPU0 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/__init__.py:134: UserWarning:
Found GPU1 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/__init__.py:134: UserWarning:
Found GPU2 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/__init__.py:134: UserWarning:
Found GPU3 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
Traceback (most recent call last):
File "train.py", line 382, in
train()
File "train.py", line 143, in train
yolact_net = Yolact()
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in __init__
self.backbone = construct_backbone(cfg.backbone)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone
backbone = cfg.type(*cfg.args)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in __init__
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in __init__
False, _pair(0), groups, bias, padding_mode)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in __init__
self.reset_parameters()
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters
init.kaiming_uniform_(self.weight, a=math.sqrt(5))
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiming_uniform_
return tensor.uniform_(-bound, bound)
RuntimeError: CUDA error: no kernel image is available for execution on the device

saisubramani on 28 Feb 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1
Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned.
the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU0 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU1 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU2 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU3 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
Traceback (most recent call last):
File "train.py", line 382, in
train()
File "train.py", line 143, in train
yolact_net = Yolact()
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init
self.backbone = construct_backbone(cfg.backbone)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone
backbone = cfg.type(*cfg.args)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init
False, _pair(0), groups, bias, padding_mode)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init
self.reset_parameters()
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters
init.kaiming_uniform_(self.weight, a=math.sqrt(5))
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiming_uniform_
return tensor.uniform_(-bound, bound)
RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

Ma-Dan on 29 Feb 2020

👍1

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1
Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned.
the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU0 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU1 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU2 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU3 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
Traceback (most recent call last):
File "train.py", line 382, in
train()
File "train.py", line 143, in train
yolact_net = Yolact()
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init
self.backbone = construct_backbone(cfg.backbone)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone
backbone = cfg.type(*cfg.args)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init
False, _pair(0), groups, bias, padding_mode)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init
self.reset_parameters()
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters
init.kaiming_uniform_(self.weight, a=math.sqrt(5))
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiming_uniform_
return tensor.uniform_(-bound, bound)
RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

thanks for the reply @Ma-Dan, I understand that , we are going to use the trained model (yolact_base_63_8000.pth) in your script. Can i know which script is used for converting the (yolact_base_63_8000.pth)------> (yolact_base_63_800.onnx) file.

saisubramani on 29 Feb 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1
Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned.
the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU0 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU1 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU2 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU3 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
Traceback (most recent call last):
File "train.py", line 382, in
train()
File "train.py", line 143, in train
yolact_net = Yolact()
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init
self.backbone = construct_backbone(cfg.backbone)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone
backbone = cfg.type(*cfg.args)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init
False, _pair(0), groups, bias, padding_mode)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init
self.reset_parameters()
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters
init.kaiming_uniform_(self.weight, a=math.sqrt(5))
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiming_uniform_
return tensor.uniform_(-bound, bound)
RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

Can you Please tell the steps for converting the yolact.pth model to .onnx model, and mention which script should be used for converting,so that it can be helpful to me. My idea is to convert the model into tensorRT, so i am trying to convert the [yolact to onnx to tensorRT].

saisubramani on 29 Feb 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1
Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

hi i am trying to start the custom training, while starting the training it shows the error, i am running the script with all dependency which you mentioned.
the Error is :

home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU0 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU1 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU2 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py:134: UserWarning:
Found GPU3 GRID K520 which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
Traceback (most recent call last):
File "train.py", line 382, inhow can
train()
File "train.py", line 143, in train
yolact_net = Yolact()
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/yolact.py", line 395, in init
self.backbone = construct_backbone(cfg.backbone)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 437, in construct_backbone
backbone = cfg.type(*cfg.args)
File "/home/ubuntu/efs_model/models/YOLACT/Modified_Yolact/backbone.py", line 64, in init
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 332, in init
False, _pair(0), groups, bias, padding_mode)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 46, in init
self.reset_parameters()
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 49, in reset_parameters
init.kaiming_uniform_(self.weight, a=math.sqrt(5))
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/nn/init.py", line 315, in kaiming_uniform_
return tensor.uniform_(-bound, bound)
RuntimeError: CUDA error: no kernel image is available for execution on the device

Hi! The modification I made is only useful when converting trained model to onnx, please use original code in your custom training.

hi i had founded that you are using eval.py script for converting the yolact model to onnx model i having a doubt
pred_outs = net(batch)
This give a list which having an size of 1, how you are using the index in ,
preds = detect({'loc': pred_outs[0], 'conf': pred_outs[1], 'mask':pred_outs[2], 'priors': pred_outs[3], 'proto': pred_outs[4]})

it showing

IndexError: list index out of range

so what i did is , i just added few lines
pred_outs = dict(pred_outs[0]) pred_outs=pred_outs['detection']

now its in dictionary format by using the key value i can take the values of detection, but when i cross checked the detection which is in (dictionary format) it having a key values of

('mask','class','score','proto','net')

what value can i assign for the

pred_out[0],pred_outs[1],pred_outs[2], pred_outs[3],pred_outs[4]

In my understanding 'conf' ':mean score,'mask':mask,'proto': means proto what about 'loc' and 'priors'

preds = detect({'loc': pred_outs[0], 'conf': pred_outs[1], 'mask':pred_outs[2], 'priors': pred_outs[3], 'proto': pred_outs[4]})

i tried like this
preds = detect({'loc': pred_outs['box'], 'conf': pred_outs['score'], 'mask':pred_outs['mask'], 'priors': pred_outs['class'], 'proto': pred_outs['proto']})
It showing Error:
TypeError: __call__() missing 1 required positional argument: 'net'
can you help me to sort this issue ? if i am wrong please tell me.@Ma-Dan

saisubramani on 29 Feb 2020

Hi,@Ma-Dan I'm trying to convert yolact model to TensorRT and facing number of issues.

Here is my working environment :

pytorch == 1.4.0
TensorRT == 7.0.0(official docker release 20.01)

There are links to yolact model in onnx format opset version == 9,

The Error is :
UNSUPPORTED_NODE: Assertion failed: scales_input.is_weights()

opset version == 11
The Error is :
INVALID_GRAPH: Assertion failed: ctx->tensors().count(inputName)

Thank you in advance for your help

AlexanderSlav on 3 Mar 2020

I have converted yolact to onnx without Detect part, and also modified some upsampling code.
https://github.com/Ma-Dan/yolact/tree/onnx
Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods.
@Ma-Dan I also converted onnx model to CoreML model, 4 custom layers need to be implemented to make it work.

Thanks for your gread job. I follow your code to convert onnx success, but convert onnx to coreml, it shows errors about upsample layer(you mentioned you modify some upsampling code, could you pls share the modification part? you men the function: def _convert_upsample(builder, node, graph, err): in /home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators.py" ):
175/308: Converting Node Type Upsample
176/308: Converting Node Type Conv
177/308: Converting Node Type Add
178/308: Converting Node Type Upsample
Traceback (most recent call last):
File "/home/jiapy/workspace/segmentation/yolact-coreml/onnx_to_coreml.py", line 15, in
minimum_ios_deployment_target='12' # TypeError: 'set' object is not callable
File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/converter.py", line 629, in convert
_convert_node(builder, node, graph, err)
File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators.py", line 2017, in _convert_node
return converter_fn(builder, node, graph, err)
File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators.py", line 1654, in _convert_upsample
input_shape = graph.shape_dict[node.inputs[0]]
KeyError: '533'

Process finished with exit code 1

dzyjjpy on 12 Mar 2020

I have converted yolact to onnx without Detect part, and also modified some upsampling code.
https://github.com/Ma-Dan/yolact/tree/onnx
Onnx model can get output of loc, conf, mask and proto, and detect process should be implemented with other methods.
I also converted onnx model to CoreML model, 4 custom layers need to be implemented to make it work.

Hi.
I'm pretty newbie of ML.
I followed Ma-Dan's work for a while, and finally I got yolact.onnx model.
But my ultimate goal is to run Unity program with yolact.
And now, I got another error dealing with onnx model.

Unexpected error while evaluating model output 783. System.ArgumentException: Cannot reshape array of size 62208 into shape with multiple of 1024 elements
at Barracuda.TensorExtensions.Reshape

I used opset=9, input=(1,550,550,3), model=resnet50-54
Someone posted simillar issues about importing onnx, but I couldn't find the exact solution.
So, if anyone had same issues or solved it, please give me a tip.

bbico on 7 May 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1

Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

Hi @Ma-Dan , Do you know how to convert yolact_plus_base_54_800000.pth to ONNX. I run $python eval.py --trained_model=weights/yolact_plus_base_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg, got Eorror:
Multiple GPUs detected! Turning off JIT.
Config not specified. Parsed yolact_plus_base_config from the file name.

Traceback (most recent call last):
File "eval.py", line 980, in
set_cfg(args.config)
File "/home/username/Document/yolact/yolact/data/config.py", line 676, in set_cfg
cfg.replace(eval(config_name))
File "", line 1, in
NameError: name 'yolact_plus_base_config' is not defined

biyuehuang on 8 Jun 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1

Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

Thank you so much @Ma-Dan , have you tried converting it to tflite?

amitkumar-delhivery on 9 Jun 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1
Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

Hi @Ma-Dan , Do you know how to convert yolact_plus_base_54_800000.pth to ONNX. I run $python eval.py --trained_model=weights/yolact_plus_base_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg, got Eorror:
Multiple GPUs detected! Turning off JIT.
Config not specified. Parsed yolact_plus_base_config from the file name.

Traceback (most recent call last):
File "eval.py", line 980, in
set_cfg(args.config)
File "/home/username/Document/yolact/yolact/data/config.py", line 676, in set_cfg
cfg.replace(eval(config_name))
File "", line 1, in
NameError: name 'yolact_plus_base_config' is not defined

give config file --config=custom_config as argument when executing eval.py as you might have provided while running the train.py!

amitkumar-delhivery on 9 Jun 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1
Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

Thank you so much @Ma-Dan , have you tried converting it to tflite?

Done :)

amitkumar-delhivery on 9 Jun 2020

The environment I used:
onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1
Run
python eval.py --trained_model=weights/yolact_darknet53_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to generate onnx file.
And run
python onnxeval.py --trained_model=weights/yolact_resnet50_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=dog.jpg
to evaluate with onnx.

Thank you so much @Ma-Dan , have you tried converting it to tflite?

Done :)

@amitkumar-delhivery were you able to convert it into tflite? if so did you used it on mobile devices(Android) ?

bmabir17 on 25 Jun 2020

@ridasalam 我将其转换为纯tensorflow，在i5 cpu上约400〜500ms。
在GPU上，pytorch和tensorflow的花费时间几乎相等。
yolact转为纯tensorflow得项目能共享吗？

18483666678 on 8 Jul 2020

RuntimeError: Only tuples, lists and Variables supported as JIT inputs/outputs. Dictionaries and strings are also accepted but their usage is not recommended. But got unsupported type Yolact

carlsummer on 7 Sep 2020

@ABlueLight you said you were successful in converting the yolact to onnx and deploy it on TensorFlow. I was wondering if you could share your code? I am still trying to figure out how to convert Yolact to ONNX and then deploy it on TensorFlow. Thanks!

h-aboutalebi on 14 Sep 2020

HI all I used @Ma-Dan repo to convert the .pth file to .onnx file. Now I want to run this using C++. What should I do next? Can someone give me a good direction? What I needed initially was to convert .pt file to .pth file but from this issue I realized that I can convert it to ONNX and then C++. Am I thinking rightly so?

Should I now look for solution to convert .onnx to .pth or .onnx can be called from C++ (in C++ implementation)?

areebsyed on 26 Oct 2020

Thank you very much for your work.
I have successfully transferred to ONNX. However, when using connxeval.py to verify the inference effed,
it is found that the starting and ending coordinates of all boxes are the same(x1=x2,y1=y2).
Draw out the image under the reference.

Can anyone give me some advice?
@Ma-Dan
000000397133

In addition, if I don't think about the above problem, I successfully converted onnx to TensorRT.
And quantified to INT8

yaoyh on 9 Dec 2020

Has anyone successfully converted a Yolact++ model to onnx? For example, yolact_plus_base_54_800000.pth to yolact_plus_base_54_800000.onnx? I'm not sure the @Ma-Dan has updated this repo to support the Yolact++ models. When I try to run eval.py as

python3 eval.py --trained_model=yolact_plus_base_54_800000.pth --score_threshold=0.3 --top_k=100 --cuda=False --image=image.jpg

I get:

Multiple GPUs detected! Turning off JIT.
Config not specified. Parsed yolact_plus_base_config from the file name.

Traceback (most recent call last):
  File "eval.py", line 990, in <module>
    set_cfg(args.config)
  File "yolact/data/config.py", line 676, in set_cfg
    cfg.replace(eval(config_name))
  File "<string>", line 1, in <module>
NameError: name 'yolact_plus_base_config' is not defined

@amitkumar-delhivery suggested above that for this error, add: --config=custom_config, but this leads to

NameError: name 'custom_config' is not defined

Any suggestions would be appreciated.

rbgreenway on 27 Jan 2021

@rbgreenway
It's look like you don't have thus conifgs.
Take a look over data/config.py if you have "yolact_plus_base_config" in it.
Also "custom_config" it's just an example for config. you should use the name you have.

sdimantsd on 28 Jan 2021

@sdimantsd
So, I basically took what I needed from the data/config.py in Ma-Dan's main branch, and put it in the onnx branch to eliminate the "yolact_plus_base_config" issue; however, now there is something very weird:

python3 eval.py --trained_model=./weights/yolact_plus_base_54_800000.pth --score_threshold=0.15 --top_k=15 --cuda=False --image=/home/bryan/Pictures/stef.jpg
Multiple GPUs detected! Turning off JIT.
Config not specified. Parsed yolact_plus_base_config from the file name.

Loading model...Traceback (most recent call last):
  File "eval.py", line 1023, in <module>
    net = Yolact()
  File "/home/bryan/python_projects/yolact_onnx/yolact/yolact.py", line 398, in __init__
    self.backbone = construct_backbone(cfg.backbone)
  File "/home/bryan/python_projects/yolact_onnx/yolact/backbone.py", line 437, in construct_backbone
    backbone = cfg.type(*cfg.args)
  File "/home/bryan/python_projects/yolact_onnx/yolact/backbone.py", line 69, in __init__
    self._make_layer(block, 64, layers[0])
  File "/home/bryan/python_projects/yolact_onnx/yolact/backbone.py", line 87, in _make_layer
    if stride != 1 or self.inplanes != planes * block.expansion:
AttributeError: 'int' object has no attribute 'expansion'

I thought it might be a version issue, but I've replicated what people have mentioned above with no change.
The environment I used:

onnx 1.4.1
onnxruntime 0.4.0
torch 1.0.1
torchvision 0.2.1

That seems like a pytorch problem, but I'm still digging....

rbgreenway on 28 Jan 2021

File "/home/anlab/anaconda3/envs/yolact-env/lib/python3.7/site-packages/coremltools/converters/onnx/_converter.py", line 600, in convert graph = _prepare_onnx_graph(onnx_model.graph, transformers, onnx_model.ir_version) File "/home/anlab/anaconda3/envs/yolact-env/lib/python3.7/site-packages/coremltools/converters/onnx/_converter.py", line 464, in _prepare_onnx_graph graph_ = graph_.transformed(transformers) File "/home/anlab/anaconda3/envs/yolact-env/lib/python3.7/site-packages/coremltools/converters/onnx/_graph.py", line 232, in transformed return _apply_graph_transformations(graph, transformers) # type: ignore File "/home/anlab/anaconda3/envs/yolact-env/lib/python3.7/site-packages/coremltools/converters/onnx/_graph.py", line 73, in _apply_graph_transformations graph = transformer(graph) File "/home/anlab/anaconda3/envs/yolact-env/lib/python3.7/site-packages/coremltools/converters/onnx/_transformers.py", line 842, in __call__ output = np.take(x, range(s, e), axis=a) # type: ignore File "<__array_function__ internals>", line 6, in take File "/home/anlab/anaconda3/envs/yolact-env/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 191, in take return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode) File "/home/anlab/anaconda3/envs/yolact-env/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc return bound(*args, **kwds) MemoryError

my computer: RAM 32gb, GeForce GTX 1060 6GB.
i have an above trouble when i attempt to convert yolact.onnx model into yolact.mlmodel. who can help me, thanks alot.

PhuowngNam on 1 Feb 2021

@Ma-Dan Thank you very much for your open source and help. I used your github to convert the model to ncnn: https://github.com/Ma-Dan/yolact/tree/onnx.
I would like to ask if you have tried to convert onnx to ncnn? Because segmentation fault (core dumped) appears when I execute ncnnoptimize.
I am thinking about whether it is related to the error message that appears when converting onnx. The following is my error message. I will trouble you again, thank you.

The environment I used: python 3.7.9, torch1.5.0, torchvision 0.6.0, onnx 1.8.1

TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if self.last_img_size != (cfg._tmp_img_w, cfg._tmp_img_h):