Pysyft: Benchmark Normal Training vs SMPC Training

[gmuraru]

What?

Benchmark on how long it takes to train a model using SMPC vs Normal training setup.

As the model we can use a simple convolutional network and as the dataset, we can use MNIST.
A good starting point can be the tutorials.

The final output should be a graph or graphs (png images) where:

• X-axis - the used precision for the FPT
• Y-axis - the time to compute the operation

We can have multiple lines, each one representing an approximation method.

Normal training time vs SMPC training time + difference in accuracy

See the Epic to check for more details.

[kamathhrishi]

I would like to take this up. So the results should be in a notebook right ?

[gmuraru]

I was thinking that we can have the training in the notebook and an image (or more) directly into the benchmark folder such that we might access it in an easy way.

[gmuraru]

@kamathhrishi how is the progress with this?

[kamathhrishi]

Going to take a while before I complete it. I will like to complete it, but if someone else is interested to do it before me please go ahead.

[arturomf94]

@gmuraru I'll take this up. I assume @kamathhrishi is not working on it anymore...

[kamathhrishi]

@arturomf94 Go ahead :)

[cereallarceny]

Hello! Just letting you know that we are no longer planning on supporting anything on the 0.2.x product line and that all work should be ported over to 0.3.x, which is considered a complete rebuild of PySyft. Because of that, I'll be closing this issue. If you feel this is a mistake, or if the issue actually applies to 0.3.x as well, please feel free to ping me on Slack and I'll reopen the issue.