Keras: Variational Autoencoder (VAE) Implementation using Keras in R

[rajafan]

I am trying to developing VAE models using keras in R. It seems that all the VAE-relevant implementations using keras are done in Python. Even though there are some example python codes, I have not got any luck in implementing them in R.

I think it would be great if there are some tutorials or example codes on this topic. Any help will be much appreciated.

Best regards
Fan

[dfalbel]

@rajafan I just started a pull request implementing a variational autoencoder in keras R. Have a look here:
https://github.com/rstudio/keras/pull/49

It's not ready yet, but only visualizations are missing :)

[rajafan]

@dfalbel , awesome, I will study the code, great thanks!

[dfalbel]

@rajafan It's now available at keras webpage too: https://rstudio.github.io/keras/articles/examples/variational_autoencoder.html

[rajshah4]

Nice job on implementing the variational autoencoder. For the visualization of the latent manifold, I find the settings in the example (and in Chollet's tutorial) don't work the best. In this example, I find changing the code to a narrower range of the latent manifold provides a better output. The current setting produced this for me:

But by making the predictions over a narrower area:

grid_x <- seq(-4, 4, length.out = n)
grid_y <- seq(-4, 4, length.out = n)

I had an output that shows the manifold in more detail:

Just curious if other people had similar results and if it makes sense changing the values in the script.

[rajshah4]

The other useful addition to the script would be showing the original sample and the reconstructed sample. Here is a quick way to do this (I am sure there are cleaner ways of doing this).

##Plot Original
sample <- 7
x_test [sample,] %>% matrix(ncol = 28) %>% as.raster() %>% plot()
##Plot Reconstructed
encoded_data <- predict(encoder, x_test[1:100,], batch_size = 100)
predict(generator, t(encoded_data[sample,])) %>% matrix(ncol = 28) %>% as.raster() %>% plot()

[rajafan]

@dfalbel @rajshah4 Thanks for the input. One minor question is about the output of the encoder, as shown in the code as follows:
"z_mean <- layer_dense(h, latent_dim)
z_log_var <- layer_dense(h, latent_dim)
"
From my understanding, it means z_mean and z_log_var are exactly the same thing, and they are both matrices with a dimension of h rows and latent_dim columns. Is it correct? Thanks.

[dfalbel]

@rajafan They are matrixes of batch_size rows and latent_dim columns. But, their values are not identical at all.

layer_dense is only stating that both will be calculated by something like h*W + b where W is a matrix of weights (256, 2) that will be estimated by the model. The weights will be estimated differently for z_mean and z_log_var.

@rajshah4 I think you can make a PR for those changes :)

[rajafan]

@dfalbel , I see, thanks for the clarification.