Hi sparklyr team,
Great job with this package, it's an amazing work.
It would be a great enhancement if is it possible to train tensor flow models via sparlyr on a spark cluster !
Would be great !
Fabien
@Fooourche it should be possible right now to do "embarrassingly parallel" hyperparameter tuning with spark_apply()
. We'll try to put some documentation together. Distributed training (training TF models on big data distributed over a cluster) is tougher and is out of scope right now for sparklyr.
Looking forward to train tensorflow on sparklyr + 1
Same here, would love to see some documentation for running tensorflow on spark through R