Ignite: Example implementation of BYOL

[vfdev-5]

🚀 Feature

It would be nice to provide BYOL implementation with Ignite. For reference, we can take this repository: https://github.com/lucidrains/byol-pytorch

[shngt]

Hi, is this still a desired feature?

[vfdev-5]

@shngt yes, it is still open.

[shngt]

The BYOL paper uses ImageNet for experiments, but it can't be downloaded via torchvision. Should I use another dataset for the example, or just implement the DataLoaders and ask the user to download the dataset on their own? Or is there some other protocol to be followed?

[sdesrozis]

It would be nice to have an ImageNet example. It should be added to reference use cases.

But maybe in a first step, we could provide an example using cifar which is available with torchvision.

What do you think ?

EDIT from https://arxiv.org/pdf/2006.07733.pdf

There are a lot of experiments based on ImageNet. I think that is what you would do. So cifar should not be sufficient...

Look here for a reference classification task from ImageNet https://github.com/pytorch/ignite/tree/master/examples/references/classification/imagenet

Dataloader is here https://github.com/pytorch/ignite/blob/915d69e8ebefdee8703f74f73af7e4c0bba0f771/examples/references/classification/imagenet/code/dataflow/dataloaders.py#L19

We suppose that ImageNet dataset is already downloaded. It is not explicitly said, we should add a mention on that point.

[shngt]

I agree - using CIFAR would not help much with ensuring correctness of the implementation. Thank you for pointing this out this file :)

[sally20921]

Please note that I am in the process of implementing BYOL and BYOLv2(SimSiam) in ImageNet using Ignite myself!
Please visit:
BYOL - https://github.com/sally20921/BYOL
BYOLv2 - https://github.com/sally20921/SimSiam

[sdesrozis]

Please note that I am in the process of implementing BYOL and BYOLv2(SimSiam) in ImageNet using Ignite myself!
Please visit:
BYOL - https://github.com/sally20921/BYOL
BYOLv2 - https://github.com/sally20921/SimSiam

@sally20921 Very excited to see your BYOL implementation powered by ignite ! I will have a look as soon as possible !