Tensorboardx: RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got numpy.ndarray

[xiongzhp]

I am pretty sure that the inputs are lists. I am not sure whether I am using tensorboard rightly.

for epoch in range(epochs):    
    batch_loss_list = []

    batch_list = seqHelper.gen_batch_list_of_lists(train_list,batch_size,(random_seed+epoch))

    #run through training minibatches
    for counter, train_batch in enumerate(batch_list):
        x_atom,x_bonds,x_atom_index,x_bond_index,x_mask,y_val = seqHelper.get_info_with_smiles_list(train_batch,\
                smiles_to_measurement,smiles_to_atom_info,smiles_to_bond_info,\
                smiles_to_atom_neighbors,smiles_to_bond_neighbors,smiles_to_atom_mask)

        atoms_prediction, mol_prediction = model(x_atom,x_bonds,x_atom_index,x_bond_index,x_mask)
        with SummaryWriter(comment='Fingerprint') as w:
            w.add_graph(model, (x_atom,x_bonds,x_atom_index,x_bond_index,x_mask))

It returns:

RuntimeError                              Traceback (most recent call last)
<ipython-input-6-fd2c7c65a5f0> in <module>()
      1 with SummaryWriter(comment='Fingerprint') as w:
----> 2     w.add_graph(model, (x_atom,x_bonds,x_atom_index,x_bond_index,x_mask))
      3 

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/tensorboardX/writer.py in add_graph(self, model, input_to_model, verbose)
    398                 print('add_graph() only supports PyTorch v0.2.')
    399                 return
--> 400         self.file_writer.add_graph(graph(model, input_to_model, verbose))
    401 
    402     @staticmethod

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/tensorboardX/graph.py in graph(model, args, verbose)
     50     if LooseVersion(torch.__version__) >= LooseVersion("0.4"):
     51         with torch.onnx.set_training(model, False):
---> 52             trace, _ = torch.jit.get_trace_graph(model, args)
     53         torch.onnx._optimize_trace(trace, False)
     54     else:

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/jit/__init__.py in get_trace_graph(f, args, kwargs, nderivs)
    251     if not isinstance(args, tuple):
    252         args = (args,)
--> 253     return LegacyTracedModule(f, nderivs=nderivs)(*args, **kwargs)
    254 
    255 

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    369             result = self._slow_forward(*input, **kwargs)
    370         else:
--> 371             result = self.forward(*input, **kwargs)
    372         for hook in self._forward_hooks.values():
    373             hook_result = hook(self, input, result)

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/jit/__init__.py in forward(self, *args)
    277     def forward(self, *args):
    278         global _tracing
--> 279         in_vars, in_desc = _flatten(args)
    280         # NOTE: use full state, because we need it for BatchNorm export
    281         # This differs from the compiler path, which doesn't support it at the moment.

RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got numpy.ndarray\

[lanpa]

As it stated, you passed a numpy array. You have to convert it first.

btw, I have not tested the graph with multiple input...

[xiongzhp]

Actually, I have converted like list(x_atom)，but it didn't work neither

[lanpa]

you need something like
x_atom = torch.floatTensor(x_atom)

[xiongzhp]

New errors, and maybe because some of my inputs are not variable. After some googling, I haven't fixed it. Any help would be appreciated!

/home/deepchem/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/onnx/utils.py:313: UserWarning: ONNX export failed on index because torch.onnx.symbolic.index does not exist
  .format(op_name, op_name))
/home/deepchem/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/onnx/utils.py:313: UserWarning: ONNX export failed on stack because torch.onnx.symbolic.stack does not exist
  .format(op_name, op_name))
/home/deepchem/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/onnx/utils.py:313: UserWarning: ONNX export failed on var because torch.onnx.symbolic.var does not exist
  .format(op_name, op_name))
/home/deepchem/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/onnx/symbolic.py:55: UserWarning: ONNX export failed on dim because ONNX and PyTorch use different strategies to split the input. not supported
  warnings.warn("ONNX export failed on " + op + " because " + msg + " not supported")
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-5-75309135b167> in <module>()
     10         atoms_prediction, mol_prediction = model(x_atom,x_bonds,x_atom_index,x_bond_index,x_mask)
     11         with SummaryWriter(comment='Fingerprint') as w:
---> 12             w.add_graph(model, (torch.Tensor(x_atom),torch.Tensor(x_bonds),torch.cuda.LongTensor(x_atom_index),torch.cuda.LongTensor(x_bond_index),torch.Tensor(x_mask),))
     13 
     14         model.zero_grad()

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/tensorboardX/writer.py in add_graph(self, model, input_to_model, verbose)
    398                 print('add_graph() only supports PyTorch v0.2.')
    399                 return
--> 400         self.file_writer.add_graph(graph(model, input_to_model, verbose))
    401 
    402     @staticmethod

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/tensorboardX/graph.py in graph(model, args, verbose)
     60     if verbose:
     61         print(graph)
---> 62     list_of_nodes = parse(graph)
     63     nodes = []
     64     for node in list_of_nodes:

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/tensorboardX/graph.py in parse(graph)
     30     nodes = []
     31     for n in graph.nodes():
---> 32         attrs = {k: n[k] for k in n.attributeNames()}
     33         attrs = str(attrs).replace("'", ' ')  # singlequote will be escaped by tensorboard
     34         if any(i.uniqueName() not in scope.keys() for i in n.inputs()):  # 0.3.1 workaround

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/tensorboardX/graph.py in <dictcomp>(.0)
     30     nodes = []
     31     for n in graph.nodes():
---> 32         attrs = {k: n[k] for k in n.attributeNames()}
     33         attrs = str(attrs).replace("'", ' ')  # singlequote will be escaped by tensorboard
     34         if any(i.uniqueName() not in scope.keys() for i in n.inputs()):  # 0.3.1 workaround

~/anaconda3/envs/deepchem/lib/python3.5/site-packages/torch/onnx/utils.py in _node_getitem(self, k)
    372     """
    373     sel = self.kindOf(k)
--> 374     return getattr(self, sel)(k)
    375 
    376 

RuntimeError: VariableType::ID() not implemented

[lanpa]

Which pytorch version are you using?

[xiongzhp]

pytorch 0.4.0

[lanpa]

HI, Can you post your model?

[xiongzhp]

https://gist.github.com/xiongzhp/aae572680555545e4b1b7757485ff6a3
This is my model, thank you so much!

[lanpa]

I also need the shape of (x_atom,x_bonds,x_atom_index,x_bond_index,x_mask) to forward the model.

Thanks

[xiongzhp]

Thank you for your kind reply, this is my repo, you can run it up directly: https://github.com/xiongzhp/molecular_structure_optimizer

[lanpa]

@xiongzhp I need ../data/hERG_ki.csv (I ran regressor.py)

[xiongzhp]

uploaded just now

[lanpa]

The constant breaks the code.

--- a/2-prediction/PytorchModelLinear.py
+++ b/2-prediction/PytorchModelLinear.py
@@ -52,7 +52,7 @@ class Fingerprint(nn.Module):

         #then normalize
         normed_features = (atom_transformed - mol_mean) / \
-                torch.sqrt(mol_var + 0.00001)
+                torch.sqrt(mol_var)

         #do nonlinearity
         activated_features = F.relu(normed_features)
@@ -81,7 +81,7 @@ class Fingerprint(nn.Module):

             #then normalize
             normed_features = (atom_transformed - mol_mean) / \
-                    torch.sqrt(mol_var + 0.00001)
+                    torch.sqrt(mol_var)

             #do nonlinearity
             activated_features = F.relu(normed_features)

[xiongzhp]

It works, but looks weird. I still need some time to figure out what's the point of the generated graph

[teffland]

For future reference, it is the use of contants that causes this issue and you can get around it by wrapping them in scalar tensors
e.g.

y = 1. - x  #breaks

y = torch.FloatTensor([1.]) - x # works fine

[ajaysg-zz]

My code is

a=torch.randn(1,80,100,requires_grad=True)
torch.onnx.export(waveglow,a, "waveglow.onnx")
I am trying to export a PyTorch model to ONNX format so i can use it in TensorRT. while testing my model in PyTorch the input tensor dimension is (1,80,x) where x varies depending on the input text length(the model i am using is TTS model named waveglow)

when i try to run the above code for exporting the model to onnx, I am always getting this error

RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got NoneType
Please help

[lanpa]

@ajaysg Looks like it's an onnx problem. I guess some layer you used is not supported by onnx yet.

[VCBE123]

My code is

a=torch.randn(1,80,100,requires_grad=True)
torch.onnx.export(waveglow,a, "waveglow.onnx")
I am trying to export a PyTorch model to ONNX format so i can use it in TensorRT. while testing my model in PyTorch the input tensor dimension is (1,80,x) where x varies depending on the input text length(the model i am using is TTS model named waveglow)

when i try to run the above code for exporting the model to onnx, I am always getting this error

RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got NoneType
Please help

I meet the same errors when i use hiddenlayer repo.

[zubairahmed-ai]

Do you know whats wrong with below code, I'm getting same error as OP

dlab = models.segmentation.deeplabv3_resnet101(pretrained=1).eval()
dummy_input = torch.randn(10, 3, 450, 450)
torch.onnx.export(dlab, dummy_input, "dlab.onnx", verbose=True)

File "D:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\jit\__init__.py", line 231, in get_trace_graph return LegacyTracedModule(f, _force_outplace, return_inputs)(*args, **kwargs)
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\jit\__init__.py", line 295, in forward
out_vars, _ = _flatten(out)
RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got collections.OrderedDict

[magneter]

@zubairahmed-ai Have you solved the problem yet?
A same error exactly occured in my project, any ideas?
Thanks.

[zubairahmed-ai]

@zubairahmed-ai Have you solved the problem yet?
A same error exactly occured in my project, any ideas?
Thanks.

Upgrade your PyTorch, also some models are not supported when exporting to ONNX, I gave up on DeepLabV3 and export MobileNetV2 though I haven't tested it yet, good luck

[magneter]

Thanks.
It`s frustrated.