Keras-yolo3: IndexError in train_bottleneck.py

[moshfiqur]

Sorry for posting this as issue as I am quite sure this is just lack of my understanding. But I am stuck with this issue and really could use the expert opinion. I have already used this YOLO3 implementation two times but didn't face any problem. Both of those time, I used a small dataset (Raccoon and Cats/Dogs) of 100s. Both time it worked great. This time I was using a larger dataset of 25000 images and I got this IndexError.

I tried to debug by myself and found that it is happening when splitting the bottleneck features into train and validation set. According to val_split, they were split as 90%/10%.

val_split = 0.1
with open(annotation_path) as f:
    lines = f.readlines()
np.random.seed(10101)
np.random.shuffle(lines)
np.random.seed(None)
num_val = int(len(lines)*val_split)
num_train = len(lines) - num_val

And this is how bottleneck features were being split by number of lines.

bottlenecks_train=[dict_bot["bot0"][:num_train], dict_bot["bot1"][:num_train], dict_bot["bot2"][:num_train]]
bottlenecks_val=[dict_bot["bot0"][num_train:], dict_bot["bot1"][num_train:], dict_bot["bot2"][num_train:]]

Traceback (most recent call last):
File "train_bottleneck.py", line 246, in bottleneck_generator
b0[b] = bottlenecks[0][i]
IndexError: index 0 is out of bounds for axis 0 with size 0

Traceback (most recent call last):
File "train_bottleneck.py", line 246, in bottleneck_generator
b0[b] = bottlenecks[0][i]
IndexError: index 224 is out of bounds for axis 0 with size 224

print(dict_bot['bot0'].shape)
print(dict_bot['bot1'].shape)
print(dict_bot['bot2'].shape)

(224, 13, 13, 1024)
(224, 26, 26, 512)
(224, 52, 52, 256)

When I print the bottleneck features shape, I found they have 224 items in the array. So, how can I train the model with a large dataset? Do I need to split the annotations into a smaller subset which contains less than 224 items in each subset? Thanks for any direction.

[kbeswick]

I ran into this issue and what solved it for me was deleting the bottlenecks.npz file so that it recalculates the bottlenecks on the next run of the script.

[moshfiqur]

@kbeswick I found my solution but I totally forgot that I posted my problem here. For my case, the problem was with images with size smaller than 224 px. So, once I removed those images with smaller size, the problem were fixed and were able to train even on existing bottlenecks.