Vision: HMDB51 Dataset mislabeling nearly all videos

[JMistele]

HMDB51 torchvision utility is mislabeling the gross majority of videos in the dataset. The following snippet should be sufficient to reproduce clear symptoms of the bug. Replacing HMDB_videos and HMDB_annotations with the appropriate directory, this snippet should run for about a minute on ~8 CPUs.

from torchvision.datasets import HMDB51
from collections import defaultdict

# Collect a single 1 frame clip from each video.
hmdb = HMDB51('/HMDB_videos/', '/HMDB_annotations/', 1, step_between_clips=10000)

class_counts = defaultdict(int)
for i in range(len(hmdb)):
    vid, aud, label = hmdb.__getitem__(i)
    class_counts[label] += 1

print(class_counts)

We would expect the class_counts dictionary to contain even counts for each of the 51 classes. Instead, we see only 31 classes even occurring, and inconsistently. The issue is worse with the validation set, wherein only 13 classes appear.

[JMistele]

I've pushed a succinct code change to hmdb51.py that resolves the issue cleanly.

see: https://github.com/pytorch/vision/pull/1240

[fmassa]

@bjuncek could you have a look at this issue? I'll check it more closely once I'm back from holidays on Aug 26th

[fmassa]

I had a look and it indeed seems to be a problem. But this problem should also exist for UCF101

[bjuncek]

@fmassa I'll take a look at the PR, and leave the review but you'll have to approve it :)

[ekosman]

I know it's not related to this issue, but I'm trying to use the HMDB51 class and get errors:

      0%|                                                                                                                                                                                                                            | 0/423 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torchvision/datasets/hmdb51.py", line 67, in __init__
    video_clips = VideoClips(video_list, frames_per_clip, step_between_clips)
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torchvision/datasets/video_utils.py", line 55, in __init__
    self._compute_frame_pts()
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torchvision/datasets/video_utils.py", line 84, in _compute_frame_pts
    for batch in dl:
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in __next__
    return self._process_data(data)
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
    data.reraise()
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torch/_utils.py", line 369, in reraise
    raise self.exc_type(msg)
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torchvision/datasets/video_utils.py", line 74, in __getitem__
    return read_video_timestamps(self.x[idx])
  File "/home/ekosman/anaconda3/envs/torch/lib/python3.7/site-packages/torchvision/io/video.py", line 252, in read_video_timestamps
    container.close()
AttributeError: 'av.container.input.InputContainer' object has no attribute 'close'

What could cause this?
I installed PyAV using:
conda install av -c conda-forge

[fmassa]

@ekosman you probably have an older version of PyAV.

Version 6.2.0 has the close method, see https://github.com/mikeboers/PyAV/pull/513.

We should probably mention that we require 6.2.0, and not older versions

[fmassa]

Fixed via https://github.com/pytorch/vision/pull/1240