Centernet: 转onnx时候遇到问题:Auto nesting doesn't know how to process an input object of type int

Created on 16 May 2019  ·  21Comments  ·  Source: xingyizhou/CenterNet

torch.onnx.export(self.model,...............

out = self.inner(*trace_inputs)

File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in __call__
result = self._slow_forward(input, *kwargs)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
result = self.forward(input, *kwargs)
File "/home/shankun.shankunwan/CenterNet/src/lib/models/networks/pose_dla_dcn.py", line 475, in forward
x = self.dla_up(x)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in __call__
result = self._slow_forward(input, *kwargs)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
result = self.forward(input, *kwargs)
File "/home/shankun.shankunwan/CenterNet/src/lib/models/networks/pose_dla_dcn.py", line 414, in forward
ida(layers, len(layers) -i - 2, len(layers))
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in __call__
result = self._slow_forward(input, *kwargs)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 464, in _slow_forward
input_vars = tuple(torch.autograd.function._iter_tensors(input))
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/autograd/function.py", line 286, in _iter
for var in _iter(o):
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/autograd/function.py", line 295, in _iter
if condition_msg else ""))
ValueError: Auto nesting doesn't know how to process an input object of type int. Accepted types: Tensors, or lists/tuples of them

在把sefl.model整体转onnx时候遇到问题，后调试时候发现，CenterNet模型中存在保存的单元？？？？这点就很怪，传统存储都是按照list，tuple，或者array和tensor，即使是单个int也应该存储到list中吧？

picture wangshankun

Most helpful comment

I modify dlav0.py line 615 - 618 to :
ret = []
for head in self.heads:
ret.append(self.getattr(head)(x))
return [ret]

and convert to onnx successes.haven't test the model yet

picture westpilgrim on 19 Jun 2019
👍3

All 21 comments

No idea.

xingyizhou picture xingyizhou on 17 May 2019

torch.onnx.export(self.model,...............

out = self.inner(*trace_inputs)

File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in call
result = self._slow_forward(input, kwargs)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
result = self.forward(input, *kwargs)
File "/home/shankun.shankunwan/CenterNet/src/lib/models/networks/pose_dla_dcn.py", line 475, in forward
x = self.dla_up(x)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in *call
result = self._slow_forward(input, kwargs)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in _slow_forward
result = self.forward(input, *kwargs)
File "/home/shankun.shankunwan/CenterNet/src/lib/models/networks/pose_dla_dcn.py", line 414, in forward
ida(layers, len(layers) -i - 2, len(layers))
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 487, in *call
result = self._slow_forward(input, *kwargs)
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 464, in _slow_forward
input_vars = tuple(torch.autograd.function._iter_tensors(input))
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/autograd/function.py", line 286, in _iter
for var in _iter(o):
File "/home/shankun.shankunwan/anaconda/envs/torch36_cuda10/lib/python3.6/site-packages/torch/autograd/function.py", line 295, in _iter
if condition_msg else ""))
ValueError: Auto nesting doesn't know how to process an input object of type int. Accepted types: Tensors, or lists/tuples of them

在把sefl.model整体转onnx时候遇到问题，后调试时候发现，CenterNet模型中存在保存的单元？？？？这点就很怪，传统存储都是按照list，tuple，或者array和tensor，即使是单个int也应该存储到list中吧？

Hello,This problems.Have you solved it?
when I run：torch_out = torch.onnx._export(model, x, "model.onnx", export_params=True)
error：
ValueError: Auto nesting doesn't know how to process an input object of type int. Accepted types: Tensors, or lists/tuples of them

my code：
model = DLASeg('dla{}'.format(34), {'hm': 1, 'wh': 2, 'reg': 2},pretrained=True,down_ratio=4,final_kernel=1,last_level=5,head_conv=256)
checkpoint = torch.load(model_path)
model.load_state_dict(checkpoint['state_dict'])
x = Variable(torch.randn(1, 3, 512, 512))
torch_out = torch.onnx._export(model, x, "model.onnx", export_params=True)

cwjhx picture cwjhx on 22 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····

def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)

wangshankun picture wangshankun on 22 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····

def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)
function.py已按照你的方式进行修改，请问DCNv2这个模型转换要怎么处理？

cwjhx picture cwjhx on 22 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····
def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)
function.py已按照你的方式进行修改，请问DCNv2这个模型转换要怎么处理？

处理不了，DCNv2是cuda c++直接写的，超出了python的跟踪能力，只能当做黑盒

wangshankun picture wangshankun on 22 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····
def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)
function.py已按照你的方式进行修改，请问DCNv2这个模型转换要怎么处理？

处理不了，DCNv2是cuda c++直接写的，超出了python的跟踪能力，只能当做黑盒

这样的话，就没办法在移动设备上部署了啊，比如TX1

qiuhaining picture qiuhaining on 24 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····
def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)
function.py已按照你的方式进行修改，请问DCNv2这个模型转换要怎么处理？

处理不了，DCNv2是cuda c++直接写的，超出了python的跟踪能力，只能当做黑盒

这样的话，就没办法在移动设备上部署了啊，比如TX1

你可以用cuda或者tensorrt重写DCNv2，做个插件进入pytorch或者trt，当然还没人这样做过，靠自己了

wangshankun picture wangshankun on 24 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····
def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)
function.py已按照你的方式进行修改，请问DCNv2这个模型转换要怎么处理？

处理不了，DCNv2是cuda c++直接写的，超出了python的跟踪能力，只能当做黑盒

这样的话，就没办法在移动设备上部署了啊，比如TX1

你可以用cuda或者tensorrt重写DCNv2，做个插件进入pytorch或者trt，当然还没人这样做过，靠自己了

或者干脆不用DCNv2，不知道结果会变差多少？论文中的resnet18就是没用DCNv2吧

qiuhaining picture qiuhaining on 24 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····

def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)
前面按照你的那个方式把int转了tensor，请问dict怎么转list啊？
我这边现在使用dlav0的模型（去掉了DCN），在模型转换时出现以下错误：
RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got dict

cwjhx picture cwjhx on 25 May 2019

@qiuhaining 这么快就high到设备上了啊？

Ai-is-light picture Ai-is-light on 26 May 2019

lib/python3.6/site-packages/torch/autograd/

我已经忘记详细报错的文件了，不过与处理int类似，写个for循环判断是dict的直接转list就好了；

wangshankun picture wangshankun on 28 May 2019

修改lib/python3.6/site-packages/torch/autograd/function.py，判断一下把所有int的转tensor存储；
修改之后还会有类似问题再dict转list，即使如果DCNv2也是没办法正常导出的，so·····

def _iter_filter(condition, allow_unknown=False, condition_msg=None,
conversion=None):
def _iter(obj):
if isinstance(obj, (int)):
print(obj)
obj = torch.tensor(obj)
前面按照你的那个方式把int转了tensor，请问dict怎么转list啊？
我这边现在使用dlav0的模型（去掉了DCN），在模型转换时出现以下错误：
RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got dict

have you solved this?could you give some sample code?

westpilgrim picture westpilgrim on 18 Jun 2019

lib/python3.6/site-packages/torch/autograd/

我已经忘记详细报错的文件了，不过与处理int类似，写个for循环判断是dict的直接转list就好了；
could you give some sample code?

westpilgrim picture westpilgrim on 19 Jun 2019

I modify dlav0.py line 615 - 618 to :
ret = []
for head in self.heads:
ret.append(self.getattr(head)(x))
return [ret]

and convert to onnx successes.haven't test the model yet

westpilgrim picture westpilgrim on 19 Jun 2019
👍3

This didn't fix the issue, can you confirm if you made any other changes?

dheerajpreddy picture dheerajpreddy on 11 Jul 2019

This didn't fix the issue, can you confirm if you made any other changes?

I changed to Msra_resnet network

westpilgrim picture westpilgrim on 9 Aug 2019
👍2

This didn't fix the issue, can you confirm if you made any other changes?

I changed to Msra_resnet network

Thanks, this works. However, this is without the DCN and hence accuracy drops :(

dheerajpreddy picture dheerajpreddy on 12 Aug 2019

Hello,
To convert to onnx, are you using msra_resnet architecture or dlav0 architecture? Without DCN?
Can you guide me with the steps, please?
Thank you!
@westpilgrim @Fighting-JJ @dheerajpreddy

mmanoharahere picture mmanoharahere on 27 Aug 2019

I modify dlav0.py line 615 - 618 to :
ret = []
for head in self.heads:
ret.append(self.getattr(head)(x))
return [ret]

and convert to onnx successes.haven't test the model yet

hi, Can be converted to onnx, which backbone network do you mean? resnet or DLA or Hourglass?thanks

zhanghongruiupup picture zhanghongruiupup on 12 Dec 2019

I modify dlav0.py line 615 - 618 to :
ret = []
for head in self.heads:
ret.append(self.getattr(head)(x))
return [ret]
and convert to onnx successes.haven't test the model yet

hi, Can be converted to onnx, which backbone network do you mean? resnet or DLA or Hourglass?thanks

is dlav0

lsccccc picture lsccccc on 14 Dec 2019

I modify dlav0.py line 615 - 618 to :
ret = []
for head in self.heads:
ret.append(self.getattr(head)(x))
return [ret]

and convert to onnx successes.haven't test the model yet

not just "dlav0.py", this fix can also be extended to other files (msra_resnet.py, resnet_dcn.py, ...). Just search for the line below:

for head in self.heads:
ret[head] = self.__getattr__(head)(x)

Got Centernet working with HiddenLayer.

markytools picture markytools on 15 Jul 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

bhack picture bhack  ·  6Comments
merckxiaan picture merckxiaan  ·  7Comments
leiyaohui picture leiyaohui  ·  3Comments
wwetnaroh picture wwetnaroh  ·  3Comments
chwong1 picture chwong1  ·  4Comments