Pytorch-lightning: How would you compare this to ignite?

[alok]

Is there a list of major differences up anywhere, to make it easier to pick between the various options?

[williamFalcon]

not yet. working on those. however, the graphic in the README should make this very clear.

At a base level, lightning implements distributed GPU, multi-node, 16-bit, auto-checkpoint and resubmit for cluster training. The learning curve with lightning is also lower since it doesn’t make unnecessary wrappings.

Further, lightning is also very thoroughly documented and well tested as is ignite.

Basically, if you want to retain 99% of the control and are ok not having to deal with your own training loop, checkpointing, distributing, etc... then lighting is for you. It’s designed to give academics the most control without oversimplifying or making a whole new library someone has to learn.

[Borda]

Hello, thank you such explanation, maybe you may consider a table with features in rows and framework in columns and check-list if the framework has it or not...
Btw, I was a bit puzzled by the very detail granulity of your releases, don't you think that the 4th (e.g. 0.3.6.8) is too much? I would have rather higher number on the 3rd place and also I would say that each commit does not have to be separate release... :)

[williamFalcon]

Yeah, that was the development stage. We were getting key features in-place. It’s stable now, we’ll batch more features and do bigger releases.

Wanted to get rid of the major issues to have good first adoption

[Borda]

I think that you can drop the sub-minor releases and keep only the minor... :)

[vfdev-5]

@williamFalcon I would be also curious to compare both libraries and understand the idea behind the creation of pytorch-lightning as an alternative to existing frameworks.

[williamFalcon]

@vfdev-5 @Borda
https://medium.com/@_willfalcon/pytorch-lightning-vs-pytorch-ignite-vs-fast-ai-61dc7480ad8a

(@vfdev-5 if i missed anything about ignite, lmk, happy to edit!)

[vfdev-5]

@williamFalcon interesting article and I agree that in all the cases user needs to learn the API of the library. Features provided by your package are interesting and some of them are also indirectly supported by Ignite without imposing the package. For example, if user would like to use horovod with pytorch and fp16 instead of nvidia/apex, he could code it and insert into the libraries frame.
Similalry, for the reproducibility, as far as I understand the package is coupled with test-tube and maybe this can be a limitation if user would like to use mlflow, polyaxon or similiar...
Anyway, I'm not familiar with your API in details.

Concerning your tables "HPC" and "Debugging" in Ignite column I would add the "checks" for tensorboard (link), Learning-rate warm-up and actually we can schedule any optimizer parameter (not only lr), Gradient inspection with tensorboard, "Nan gradient inspection" - we have TerminateOnNan handler, "Train/Validate on subset of data" it is up to user to define and it is posible too in ignite.

Other features:

• custom events : e.g. custom epoch as a number of iterations
• out-of-box metrics for classification, segmentation, regression

So if you could include this, it will be cool :)

And definitly, if we could have a guy like you to communicate about Ignite :)

[Borda]

I would also tend to use mlflow or polyaxon intead test-tube since they seem to have much wider/larger support... :)

[williamFalcon]

Makes sense. In keeping with allowing as much flexibility as possible for researchers, let's add support for any logger they want to use.

See #47

[williamFalcon]

On a side-note, @vfdev-5 any interest in just merging ignite with Lightning? You guys could bring in some of the things you've been working on into Lightning.

That way we can all focus on fixing the reproducibility issue.

[vfdev-5]

@williamFalcon I would prefer to speak about merging lighting with Ignite :)
As the latter is older and used by at least 105 projects according to github (e.g https://medium.com/huggingface/how-to-build-a-state-of-the-art-conversational-ai-with-transfer-learning-2d818ac26313).

If you are interested, we can discuss about this in pytorch slack in pytorch-ignite channel in details.

[williamFalcon]

No worries, i prefer to keep both libraries separate then. Researchers can pick whichever makes sense for them :)