Keras: Support for Sparse Matrices

[gsimchoni]

Currently the x object accepted by the fit function is a

Vector, matrix, or array of training data...

It would be great if there is also support for sparse matrices, as these X matrices can grow very large on disk, although they can be very sparse. For example, the Nietzsche example runs fine on my 32GB RAM laptop, but I run into RAM problems even with twice as many rows. And this is a classic case for a sparse matrix.

[jjallaire]

Moved to https://github.com/rstudio/reticulate/issues/72

[dfalbel]

I guess you would need a sparseArray implementation also for this example.

While there's no support for sparse matrixes you can always use train_on_batch function. And transform only one batch per time so RAM doesn't explode. You could, instead of vectorizing evrything,
change this call:

model %>% fit(
    X, y,
    batch_size = 128,
    epochs = 1
  )

for

  all_samples <- 1:length(dataset$sentece)
  batch_size <- 128
  num_steps <- trunc(length(dataset$sentece)/batch_size)

  for(step in 1:num_steps){

    batch <- sample(all_samples, batch_size)
    all_samples <- all_samples[-batch]

    sentences <- dataset$sentece[batch]
    next_chars <- dataset$next_char[batch]

    # vectorization
    X <- array(0, dim = c(batch_size, maxlen, length(chars)))
    y <- array(0, dim = c(batch_size, length(chars)))

    for(i in 1:batch_size){

      X[i,,] <- sapply(chars, function(x){
        as.integer(x == sentences[[i]])
      })

      y[i,] <- as.integer(chars == next_chars[[i]])

    }

    model %>% train_on_batch(
      X, y
    )

  }

[jjallaire]

fit_generator() is now able to work with a custom R function so that would be another way to do this that wouldn't require writing your own training loop. Documentation is here: https://rstudio.github.io/keras/articles/faq.html#how-can-i-use-keras-with-datasets-that-dont-fit-in-memory