Tensorboardx: TypeError: 'GraphDef' object does not support indexing

[Fangyh09]

I used nested model.
Doesn't it support nested model?

[boyurmit]

I meet the same issue.
The command torch.jit.get_trace_graph(model, args) in function _graph_ of pytorch_graph.py except RuntimeError.

[lanpa]

Hi, can you provide some code to reproduce?

[SystemErrorWang]

Same problem here, below is the error log:

Error occurs, No graph saved
Checking if it's onnx problem...
Your model fails onnx too, please report to onnx team

Traceback (most recent call last):
File "/home/wangxinrui/FasterRcnn/train.py", line 134, in
train()
File "/home/wangxinrui/FasterRcnn/train.py", line 100, in train
writer.add_graph(trainer, [img, bbox, label, scale])
File "/home/wangxinrui/anaconda3/lib/python3.6/site-packages/tensorboardX/writer.py", line 520, in add_graph
self.file_writer.add_graph(graph(model, input_to_model, verbose))
File "/home/wangxinrui/anaconda3/lib/python3.6/site-packages/tensorboardX/writer.py", line 107, in add_graph
graph = graph_profile[0]
TypeError: 'GraphDef' object does not support indexing

Process finished with exit code 1

[Mojomotte]

I get the exact some error when using a nested model. Any solutions so far?

[lc17721825]

hi lanpa, thanks for your great tensorboardx. when I add_graph a nested model, I meet the same problem as SystemErrorWang. Is that something wrong with nested model? Thanks in advance.@lanpa

[ez4lionky]

I also got the same issue, when I used tensorboardX to visualize the model that the inputs have Graph(custom object) type list. Maybe oonx don't support the custom type inputs?

[nmakes]

Same issue here as well. I'm having a nested model and getting this error. My model contains a resnet18 with a tied decoder (all conv_transpose weights of the decoder are tied to the conv layers of resnet18). I have defined the conv_transpose block as follows. It takes the corresponding conv layer to get the information about weights, stride etc.:

`
class InvConv2d(nn.Module):

def __init__(self, conv_layer, stride = None, padding = None, output_padding = (0, 0)):

    super(InvConv2d, self).__init__()

    self.conv = conv_layer
    if stride == None:
        self.stride = self.conv.stride
    else:
        self.stride = stride
    if padding == None:
        self.padding = self.conv.padding
    else:
        self.padding = padding
    self.output_padding = output_padding

def forward(self, x):
    ret = F.conv_transpose2d(x, weight = self.conv.weight, bias = None, stride = self.stride, padding = self.padding, output_padding = self.output_padding)
    return ret

`

I am using these transpose convolutions in my weight - tied decoder network. I need to visualize the graph to see whether the activations are being passed correctly.

Any fix for this yet?

[rahulptel]

I am also facing the same issue while trying to visualize a nested model.

[shiranD]

I got the same error. In my case I was using
add_graph(model, input)
, and I suspect the problem was because I sent an integer and not a tensor (the input tuple aside from the input tensors also contained an integer). When casting the integer as a tensor didn't work since the code detected that I need it to be an integer in the forward pass of the model. Any chance that it will allow for ints as well?

[lanpa]

@shiranD Does x = model(input) runs without problem?

[shiranD]

This input seems to be producing a graph or at least not giving any error. Now the problem is that it refuses to display it on the board. I tried the demo_graph and it produces a graph for the model in the demo on the tensor board display. Thanks for responding so quickly!

[hanamizukigakki]

Same problem here, below is the error log:

Error occurs, No graph saved
Checking if it's onnx problem...
Your model fails onnx too, please report to onnx team

Traceback (most recent call last):
File "/home/wangxinrui/FasterRcnn/train.py", line 134, in
train()
File "/home/wangxinrui/FasterRcnn/train.py", line 100, in train
writer.add_graph(trainer, [img, bbox, label, scale])
File "/home/wangxinrui/anaconda3/lib/python3.6/site-packages/tensorboardX/writer.py", line 520, in add_graph
self.file_writer.add_graph(graph(model, input_to_model, verbose))
File "/home/wangxinrui/anaconda3/lib/python3.6/site-packages/tensorboardX/writer.py", line 107, in add_graph
graph = graph_profile[0]
TypeError: 'GraphDef' object does not support indexing

Process finished with exit code 1

I have got the same problem, if any advices?

[1zgh]

Your model cannot be exported by onnx, please report to onnx team
w.add_graph(net, (x))
File "/home/zhang/.conda/envs/torch0.4-caffe/lib/python3.6/site-packages/tensorboardX/writer.py", line 697, in add_graph
self._get_file_writer().add_graph(graph(model, input_to_model, verbose, **kwargs))
File "/home/zhang/.conda/envs/torch0.4-caffe/lib/python3.6/site-packages/tensorboardX/writer.py", line 137, in add_graph
graph = graph_profile[0]
TypeError: 'GraphDef' object does not support indexing

I have the same problem,How can I do?

[lanpa]

@1zgh You might need to update your torch version as well as tensorboardX.

[cheerss]

Aha! I have met the same problem as you but I have solved it now.

Version

I use pytorch1.1.0 and tensorboardx 1.17

Problem

I met the following error when I called "SummaryWriter().add_graph()"

Error occurs, No graph saved
Checking if it's onnx problem...
Your model cannot be exported by onnx, please report to onnx team
Traceback (most recent call last):
  File "train.py", line 96, in <module>
    main()
  File "train.py", line 38, in main
    writer.add_graph(model, (fake_input,), verbose=True, operator_export_type="RAW")
  File "/usr/local/lib/python3.6/dist-packages/tensorboardX/writer.py", line 697, in add_graph
    self._get_file_writer().add_graph(graph(model, input_to_model, verbose, **kwargs))
  File "/usr/local/lib/python3.6/dist-packages/tensorboardX/writer.py", line 137, in add_graph
    graph = graph_profile[0]
TypeError: 'GraphDef' object does not support indexing

How do I position the problem

Who prints the information?

Following the content of SummaryWriter.add_graph(), I find that:

Only one file in tensorboardx includes this line "Your model cannot be exported by onnx, please report to onnx team", that is "tensorboardx/pytorch_graph.py" (about the 286th line or so). The correspoding code are as follows:

    with torch.onnx.set_training(model, False):
        try:
            trace, _ = torch.jit.get_trace_graph(model, args)
        except RuntimeError:
            print('Error occurs, No graph saved')
            print("Checking if it's onnx problem...")
            try:
                import tempfile
                torch.onnx.export(
                    model, args, tempfile.TemporaryFile(), verbose=True)
            except RuntimeError:
                print("Your model cannot be exported by onnx, please report to onnx team")
            # Create an object matching
            # https://github.com/tensorflow/tensorboard/blob/master/tensorboard/compat/proto/graph.proto
            # The producer version has been reverse engineered from standard
            # TensorBoard logged data.
            return GraphDef(versions=VersionDef(producer=22))

As we see, the source of the problem is torch.jit.get_trace_graph(model, args) throws an exception, but it is not printed. So I try to display the error information by:

    with torch.onnx.set_training(model, False):
        try:
            trace, _ = torch.jit.get_trace_graph(model, args)
        except RuntimeError as e:
            print(e) ## !!! Add this line, actually, the code in master branch has added it
            print('Error occurs, No graph saved')
            print("Checking if it's onnx problem...")

Finally, I found the error thrown by torch.jit.get_trace_graph(model, args):

Only tuples, lists and Variables supported as JIT inputs, but got numpy.ndarray

OR

Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

OR SOMETHING ELSE

which tells us that the type of input must match the type of the model

How to fix

You cannot use numpy array as the input, you'd better to use torch.Tensor just in case. Then, if your model is on GPU, the input should on GPU, too.

Questions

1. I do not know the exactly relation between onnx and pytorch, but the error is incured by a pytorch model, why does it say: "Your model cannot be exported by onnx, please report to onnx team"

[lanpa]

@cheerss
pytorch model can be exported to onnx. And before tensorboardX 1.7, it relies on the exported onnx graph to draw on tensorboard.
And now the onnx used in add_graph have replaced with JIT since https://github.com/lanpa/tensorboardX/commit/b59a8a0237eaaeff8cb82e1d0a06c5bcac105b9b

This should be less problematic.

update:
please try v1.8 directly :)

[seenitall]

@lanpa -- same here: tensorboardX v1.8, torch 1.1.0, torchvision 0.3, model=DeepLabV3 (torchvision), model_input=torch.cuda.FloatTensor (torch.float32)

[lanpa]

@seenitall Can you post the whole error message?

[lanpa]

Close this since this thread is pretty old, and many context is outdated. Please open an issue and follow the issue template if needed.

[Devin879176]

Error occurs, No graph saved
Checking if it's onnx problem...
Your model fails onnx too, please report to onnx team
Traceback (most recent call last):
File "test.py", line 133, in
prefetch_test(opt)
File "test.py", line 75, in prefetch_test
ret = detector.run(pre_processed_images)
File "/home/devin/Project/Pycharm/CenterNet/CenterNet/src/lib/detectors/base_detector.py", line 116, in run
output, dets, forward_time = self.process(images, return_time=True)
I get the some exact problem, how do you solve it?

File "/home/devin/Project/Pycharm/CenterNet/CenterNet/src/lib/detectors/multi_pose.py", line 51, in process
w.add_graph(self.model, dummy_input)
File "/home/devin/anaconda3/envs/CenterNet/lib/python3.6/site-packages/tensorboardX/writer.py", line 520, in add_graph
self.file_writer.add_graph(graph(model, input_to_model, verbose))
File "/home/devin/anaconda3/envs/CenterNet/lib/python3.6/site-packages/tensorboardX/writer.py", line 107, in add_graph
graph = graph_profile[0]
TypeError: 'GraphDef' object does not support indexing