Deepreg: Demo: MICCAI Educational Challenge 2020

[NMontanaBrown]

Issue description

Porting docs to a condensed jupyter notebook/google colab (GPU access) for submission on top of repo to MICCAI edu challenge. The idea being that it works as a standalone introduction/tutorial for those in the MICCAI community + showcases the repo.

Useful link: https://miccai-sb.github.io/challenge

TODOs/Plan:

• [x] Condense tutorial_registration.md, quick_start.md.
  
  
  
  • [x] Edit for flow
  
  
• [x] Add demo of classical registration. Adapt Yipengs SITs, refactor?
• [x] Done, but scrapped. Build function to perform GoICP, find some interesting point clouds.
• [x] Add demo of medical image registration - adapt Shaheer's demo?
  
  
  
  • [x] A bug with the config... @mathpluscode to investigate. Changed to paired CT reg.
  
  
• [x] Add references to data sources.
• [ ] Instructions to run on azure if no access to colab Mention azure as an option if no colab.
• [x] Add "how to" ensure using GPU for runs.
• [x] Add pretrained model.
• [x] References for text
• [x] Circulate
• [x] Clarify "classical reg" - @YipengHu can you add this text, a paragraph?
• [x] Add visualisation of label based losses
  
  
  
  • Load the data for the first example prior to the explanations so we can introduce a code cell which shows the labelled losses using real medical data.
  
  
• [x] Add visualisation of intensity based losses
• [x] Add visualisation of deformation based losses
• [x] Add docs for layer, resample, grids which are used extensively in demo.
• [x] Reformat code for style consistency + comments.
• [x] Check predict update or not?
• [x] Recirculate.
• [x] Clear all outputs
• [ ] Submit 25th July 2020. Deadline extended to 10th Aug, address reviewers comments and submit early next week.

Type of Issue

Please delete options that are not relevant.

• [x] New feature request
• [x] Documentation update

[ebonmati]

Should we provide a Medium post too?

[YipengHu]

Should we provide a Medium post too?

Why not, if you are up for it!

[NMontanaBrown]

Hi @YipengHu @mathpluscode @zacbaum, I would highly appreciate some feedback on this piece of work 😄 especially on how easy to follow, appropriate level (Masters, 1st year PhD), missing text, what to expand (feel free to expand) what to ignore (add comemnt to delete).... As you see fit

[YipengHu]

you might want to acknowledge learn2reg upfront, which is more scientific and could be challenging for beginners.

[NMontanaBrown]

Hi @s-sd, thank you so much for helping getting the demo working! Now that we have a few more days, could you train a model to upload to the model zoo for longer, if it's not in training already? I will be off for a couple of days on an emergency, but could you confer with @YipengHu @mathpluscode or someone else on the team to organise this for the challenge submission? Thanks 😄

[s-sd]

@NMontanaBrown, no problem, happy to help. I'll definitely update the model zoo either tomorrow or on Wednesday. I will also have to change the notebook slightly to load in the new checkpoint so I'll leave a comment where I make the edits so the changes can be reviewed before submission. I'll comment here when I'm done with the updated notebook so that you, @mathpluscode or @YipengHu can take a look.

[YipengHu]

@s-sd if you are updating the notebook, would you mind working with 196-yh which should have the latest version. Thanks!

[s-sd]

@YipengHu sure, I'll work with the notebook from 196-yh

[s-sd]

@YipengHu, @mathpluscode I have updated the notebook (added new weights, and modified the demo visualisation code to work with updated predict) and have left it in the 196-yh branch. Should I update it in the 196-edu-challenge branch as well or is it okay to leave it in 196-yh?

Also, should I update the visualisation in the demo as well since the predict function was modified or is it okay since a the last working commit id has been supplied? This is a quick fix and the code is already in the notebook and has been run. Let me know what might be better to do.

[YipengHu]

@YipengHu, @mathpluscode I have updated the notebook (added new weights, and modified the demo visualisation code to work with updated predict) and have left it in the 196-yh branch. Should I update it in the 196-edu-challenge branch as well or is it okay to leave it in 196-yh?

Also, should I update the visualisation in the demo as well since the predict function was modified or is it okay since a the last working commit id has been supplied? This is a quick fix and the code is already in the notebook and has been run. Let me know what might be better to do.

@s-sd leaving it at 196-yh is fine and yes please update to the newest predict function after pulling the master.

[s-sd]

@YipengHu, thanks! I'll open a new issue for the demo update and create a new branch for it

[NMontanaBrown]

Thanks @s-sd @YipengHu for the work this week, and thanks to @tvercaut for the thorough comments 😄 ! Couple of thoughts about points/TODOs to address on my behalf, discussion appreciated:

1. We talk about losses a lot but don't really explain/illustrate how they work, which @tvercaut raised. Add a cell or two to illustrate. learn2reg has some really nice visualisations for deformation fields.
2. We use the warp/resample from DeepReg quite liberally without alluding to how they work (which is probably confusing), think it's worth adding comments/porting docs directly instead of pointing to docs.
3. @mathpluscode @s-sd was the predict updated here? I know there was some work done on this this past week but unsure if it's been changed or not. Are we planning on adding this?
4. @tvercaut made some valid comments about classical registration which currently isn't handled by DeepReg (currently, @mathpluscode was this a CLI in works or not?). Initially I had worked on an example with @MattClarkson's python wrapped GoICP but we don't really illustrate point set registration in the current examples either so not sure that this would make sense...
   a). Can we point to other tutorials/packages that cover this for comparison?
   b). Is it feasible to build an example to compare with each demo, is there time, relevance?
5. Re pip installing the package - not sure we can deploy this in time to PyPi for submission. But in the works.

[YipengHu]

Thanks @s-sd @YipengHu for the work this week, and thanks to @tvercaut for the thorough comments 😄 ! Couple of thoughts about points/TODOs to address on my behalf, discussion appreciated:

1. We talk about losses a lot but don't really explain/illustrate how they work, which @tvercaut raised. Add a cell or two to illustrate. learn2reg has some really nice visualisations for deformation fields.
2. We use the warp/resample from DeepReg quite liberally without alluding to how they work (which is probably confusing), think it's worth adding comments/porting docs directly instead of pointing to docs.
3. @mathpluscode @s-sd was the predict updated here? I know there was some work done on this this past week but unsure if it's been changed or not. Are we planning on adding this?
4. @tvercaut made some valid comments about classical registration which currently isn't handled by DeepReg (currently, @mathpluscode was this a CLI in works or not?). Initially I had worked on an example with @MattClarkson's python wrapped GoICP but we don't really illustrate point set registration in the current examples either so not sure that this would make sense...
   a). Can we point to other tutorials/packages that cover this for comparison?
   b). Is it feasible to build an example to compare with each demo, is there time, relevance?
5. Re pip installing the package - not sure we can deploy this in time to PyPi for submission. But in the works.

3 - 'predict' will be improved a bit to save space, but the functionalities are all there and currently save everything we need ;)
4 - CLI for classical registration isn't planned yet. let's see if there is a need, we can add post R1. I think the challenge has a slightly different view on linking the classical reg algorithms (using DeepReg functions) with learning-based ones. This is not necessirily to be reflected by the core DeepReg utilities - in theory we can even do this with feature-based registration by replacing input images with feature-representing masks (we can mention this but implementing examples with or without other packages etc would be an overkill for me). The same for point-based registration by using point spread function processed masks. Let's emphasise this a bit in the notebook.

[NMontanaBrown]

@zacbaum could you review the latest version pushed on this branch?

[zacbaum]

@zacbaum could you review the latest version pushed on this branch?

Done! Corrected several typos/misspellings and made formatting consistent.

[NMontanaBrown]

@s-sd, for the last demo, are there the same images that can be displayed in the same way as the predictions? Could you add this to the last code cell for the sake of comparison?

[s-sd]

@NMontanaBrown added the images to the plots now

[s-sd]

@NMontanaBrown @YipengHu I've added the longer trained model to the model zoo and have loaded it into the notebook. The results look visibly better than before. I have left the updated notebook in the 196-edu-challenge branch