Keras: BinaryWeight-Networks

[Cospel]

I found this paper yesterday about Binary Networks/Convolutions and XNOR-NET: http://arxiv.org/pdf/1603.05279v1.pdf (XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks).

Anyone tried this to implement in Keras/Tensorflow?

The memory savings looks great. Probably it will be possible just to implement something like Convolution2DBinary layer.

[NasenSpray]

There's no easy way to implement this in Keras w/o backend support, i.e., the required ops have to be supported by Theano/TensorFlow first

[bhack]

@Cospel Please try to open an issue upstream for ops then reference here.

[bhack]

I opened an issue on TF at https://github.com/tensorflow/tensorflow/issues/1592. I don't know what is the state of TH.

[Cospel]

Thanks @bhack, i think this could be another great feature. :)

[EderSantana]

there are some theano ops here for +1, -1 https://github.com/MatthieuCourbariaux/BinaryNet

[bhack]

@EderSantana But has a custom kernels not integrated upstream.

[EderSantana]

and the license is GPL. Theano is BSD... They won't merge it upstream

[bhack]

If you want you can try to open an issue on Theano.

[nouiz]

I just asked the author and he agree to change the license. Opening an
issue on Theano is a good idea. I let the a user that want this do it.

On Wed, Mar 23, 2016 at 10:23 AM, bhack [email protected] wrote:

If you want you can try to open an issue to Theano.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/issues/2019#issuecomment-200366087

[nouiz]

It is changed.

On Wed, Mar 23, 2016 at 10:38 AM, Frédéric Bastien <
[email protected]> wrote:

I just asked the author and he agree to change the license. Opening an
issue on Theano is a good idea. I let the a user that want this do it.

On Wed, Mar 23, 2016 at 10:23 AM, bhack [email protected] wrote:

If you want you can try to open an issue to Theano.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/issues/2019#issuecomment-200366087

[bhack]

I think that some implementation stuffs related to the paper are done in Torch and available here https://github.com/mrastegari/Deeper_Net

[Seraphli]

@bhack The repo disappeared. That's strange. I think he is the author of the paper about XNOR-Net. I was trying to find more about xnor-net, and just find this thread.

[GengCauWong]

@bhack Do you have downloaded the code of XNOR-Net from https://github.com/mrastegari/Deeper_Net ？Can you email this code to me? [email protected] Thanks!

[albicasty]

@xxlylf @bhack I'm also interested, thanks!

[bhack]

Sorry I don't have it. Please follow also the last messages from https://github.com/tensorflow/tensorflow/issues/1592

[kaihuchen]

Has there been any progress reported in making the Binary Networks available in Keras or Tensorflow?
My interest here is in using it to investigate a kind of Monte Carlo method for neural networks.
The idea here is that if the forward pass is lightning fast then it is perhaps possible to dispose of the more expensive backward passes by doing many guided guesses in the forward passes. It should be really interesting to try this out.

[xylcbd]

follow it

[bentaylordata]

Following

[DingKe]

There is an ineffient implementation for demo: https://github.com/DingKe/nn_playground/tree/master/binarynet

[ppwwyyxx]

There are some other "inefficient" implementation in tensorflow, e.g.:
DoReFa-Net: https://github.com/ppwwyyxx/tensorpack/tree/master/examples/DoReFa-Net
Binary-Weight network (with full precision activations) is a special case in DoReFa-Net.
Trained Ternary Quantization (ICLR17 submission): https://github.com/czhu95/ternarynet

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs, but feel free to re-open it if needed.

[JingrenChen]

There is an implementation of Binarized Neural Networks on embedded device:
https://gitlab.com/htkung/ddnn/

Here is the paper:
http://www.eecs.harvard.edu/~htk/publication/2017-ewsn-mcdanel-teerapittayanon-kung.pdf

[itransvideo]

following it. I want to use the binaryConnect too

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[lgeiger]

If anyone is still looking for binarized and other extremely quantized layers and pretrained models, feel free to checkout http://larq.dev and https://github.com/larq/larq which is based on tf.keras with full support for TF 1.x and 2.0.0.