Stable-baselines: Upgrade to Tensorflow 2

[iirekm]

Currently I have a huge dilemma:

• backport all my code to TF 1, in order to use Stable Baselines and my code in one project
• or use something less mature than Stable Baselines (eg TF Agents) only because I have Tensorflow 2.3 code in my project

Couldn't you support also newer TF versions (tensorflow>=1.8.0,<2.0.0 could become tensorflow>=1.8.0).

[Miffyli]

Related #366 and #733

TL;DR: Based on users' feedback we decided to continue development with PyTorch backend (stable-baselines3). Updating this repository to support TF2 would be quite a bit of work (draft here) and it has not received much attention.

I suggest using environments to handle dependencies between projects so you avoid issues like this.

[araffin]

There is an non official version with tf2 support here if you want : https://github.com/hill-a/stable-baselines/issues/984

[nbro]

TensorFlow 2 definitely needs to be supported. Nobody is going to use TF 1, if you have the opportunity to use TF 2.

[araffin]

TensorFlow 2 definitely needs to be supported. Nobody is going to use TF 1, if you have the opportunity to use TF 2.

@nbro you should probably take a look at https://github.com/hill-a/stable-baselines/issues/366 and https://github.com/hill-a/stable-baselines/issues/733

[nbro]

@araffin I already quickly looked at the first issue. As I said somewhere else, it seems that there aren't big plans to support TF 2 (https://github.com/Stable-Baselines-Team/stable-baselines-tf2) and you are already developing the new version of this package with PyTorch, which, as I said, is a risky change (https://github.com/DLR-RM/stable-baselines3) given that TF seems to be a lot more used than PyTorch, although PyTorch is also used in research and, apparently, most people voted for PyTorch in your poll. For example, now, I would need to use TF 2 with stable-baselines and I can't. I am sure there will be other people with my same issue (given that TF 2 already quite mature and we are already at version 2.3).

[araffin]

given that TF seems to be a lot more used than PyTorch, although PyTorch is also used in research and, apparently, most people voted for PyTorch in your poll.

We give more details in #733 but we chose it for mainly two reasons: maintainers choice and community. Without maintainers (and will), it is harder to develop a project. As we aim mainly at researchers and students, I think it is a conscious choice.

For example, now, I would need to use TF 2 with stable-baselines and I can't.

Actually, you can use the fork mentioned in the doc (projects section) or use the simple trick mentioned in https://github.com/araffin/rl-baselines-zoo/issues/109#issuecomment-703634580 (replacing the tf import by import tensorflow.compat.v1 as tf, it should work for most of the lib, except the part where the tf contrib is used)

, it seems that there aren't big plans to support TF 2

If you (or others) want to support and maintain a TF2 version, we would be also happy about it. But I have to warn you that it is quite time consuming.