Deepreg: Enhancement + docs: visualisation tool for demos and to demo registration on README.md

[NMontanaBrown]

Issue description

It would be powerful to be able to visualise registration results achieved with the package. At the moment I think it would be extra nice if this is showcased on the README with a GIF of some sort to show updates of learning process visually for some trained network.

Type of Issue

Please delete options that are not relevant.

• [x] New feature request
• [x] Documentation update

What's the expected result?

• Tool to plot results, maybe a keras style callback during training?
• A nice diagram/GIF/pictures of registration achieved using the package from a demo on the README to visually showcase and add some flair/impact.

[YipengHu]

@NMontanaBrown
*keras style callback during training" sounds too much to me, for R1 anyway... maybe saving to files? an item for discssion?
*pictures on readme or even frontcover sounds a great idea! @zacbaum

[zacbaum]

*pictures on readme or even frontcover sounds a great idea! @zacbaum

Perhaps gifs or photos of progress / results from the demos can be utilized? If so, please suggest which to do this for and I can incorporate it with updates that I will make for #161.

[NMontanaBrown]

I potentially have a ready built tool that can plot things during training? Let me check if it would be adaptable to the tools at hand without going too crazy.

[NMontanaBrown]

TODOs:

• [ ] Write callback/adapt callback for visualisation of reg during training
• [ ] Modify train script such that it takes this new callback. Does this warrant a separate tickets?
• [ ] Train a net with this callback.
• [ ] Download images.
• [ ] Compile into a GIF.

[zacbaum]

• [ ] Modify train script such that it takes this new callback. Does this warrant a separate tickets?

Perhaps its own (very simple) demo? One to illustrate how to use callbacks / show training progress?

[mathpluscode]

Is there any showcase of the callbacks you mean @NMontanaBrown

I'm wondering do we need plot things during training? as you can always do inference using the saved checkpoints.

My concern is mainly about the training speed.

One more point, im about to refactor train.py maybe hold on after the refactoring? What do you think?

Another point, I will list what we have as a results of prediction, maybe we can see what is needed for post-training inference, then we decide if some of them can be useful during the training.

[NMontanaBrown]

@mathpluscode for example, I log my losses graphically as .pngs during training and port them to local. Or preds from val tensor by saving as .png and again porting to local. Speed effect will depend on the ops for the callbacks - if you are asking for one pred the op will be meaningless but if you are asking for preds for many many tensors... It will depend strictly on usage! EDIT: Particularly useful on GPU where you can only see a log of losses - helps with efficiency: kill processes that in theory are optimizing but maybe a weird local minima.

I agree we will have to wait until train is refactored, can't pass a new callback if there is not option to add it to train script as is.

@zacbaum I think this is a good point - the building of this callback could be documented more thoroughly in a demo to show the process and facilitate independent usage by users 😄 .

[YipengHu]

@NMontanaBrown I would recommend to close this ticket if no further plan on this, especially given the refactored predict will have comprehensive visualisation saved in files.

[YipengHu]

I'm marking this post-r1, but feel free to assign yourself.

[acasamitjana]

If you think it is necessary, I can try to build a small script to generate videos from the output dir in demo_predict.py (already done so, just need to do it generally)

[mathpluscode]

If you think it is necessary, I can try to build a small script to generate videos from the output dir in demo_predict.py (already done so, just need to do it generally)

maybe make it applicable to any output usingpredict? But demo_predict.py is probably the same thing.