Pytorch_geometric: Pointnet2_segmentation.py: Shapenet not processing

[Tojens]

🐛 Bug

When running the pointnet2_segmentation.py example, the shapenet dataset downloads just fine, but stops after the download and creating an empty processing folder.

The traceback is as follows:
Processing... Traceback (most recent call last): File "pointnet2_segmentation.py", line 23, in <module> pre_transform=pre_transform) File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\site-packages\torch_geometric\datasets\shapenet.py", line 104, in __init__ pre_filter) File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\site-packages\torch_geometric\data\in_memory_dataset.py", line 52, in __init__ pre_filter) File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\site-packages\torch_geometric\data\dataset.py", line 102, in __init__ self._process() File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\site-packages\torch_geometric\data\dataset.py", line 154, in _process self.process() File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\site-packages\torch_geometric\datasets\shapenet.py", line 182, in process torch.save(self.collate(data_list), self.processed_paths[i]) File "C:\Users\user\AppData\Local\Continuum\anaconda3\lib\site-packages\torch_geometric\data\in_memory_dataset.py", line 125, in collate keys = data_list[0].keys IndexError: list index out of range

Expected behavior

The scripts should run without issues

Environment

• OS: Win10
• Python version: 3.7
• PyTorch version: 1.4
• CUDA/cuDNN version: 10.0

I can see there's been made a change to shapenet.py in the past month, maybe that has something to do with it?

Hope someone can be helpful.

Best regards

[rusty1s]

I will look into this. Sorry for the inconveniences!

[cristi191096]

I have encountered the same issue and looked inside the ShapeNet class.
The problem is here:
with open(path, 'r') as f: filenames = [ osp.sep.join(name.split(osp.sep)[1:]) + '.txt' for name in json.load(f) ] # Removing first directory.

name.split(osp.sep) returns a list with only one element so name.split(osp.sep)[1:] is an empty list.
I have hardcoded the value with name.split('/') and it seems it works just fine.

Hope this helps.

[rusty1s]

Why it is an empty list? For me, it says something like

shape_data/04379243/9b3433ca11cd09bae7c7920f6a65a54d

[cristi191096]

When splitting it returns something like this:
["shape_data/04379243/9b3433ca11cd09bae7c7920f6a65a54d", None, None ]

I believe the problem is when you discard the element at index 0 with name.split(osp.sep)[1:] it will discard the whole path, not just shape_data.

It might be an operating system issue with osp.sep. I run this code on Windows 10 as well. What I think it happens is that osp.sep = "\\" or "//" on Windows and couldn't split the path.

[rusty1s]

That‘s a good point. I will fix that.

[rusty1s]

Should be fixed in master.

[cristi191096]

I'm very happy I could help.

Best Wishes.