Stable-baselines: Dueling and Double DQN

[edwinlima]

Hi,
I would like to be able to compare the different variations of DQN, the vanilla DQN(original paper) and double DQN and dueling extensions but I don't know how to disable the dueling and double extensions. I notice that the double option is within the build_training method but the parameter is not exposed. Likewise the dueling option is part of the policy parameters but the DQN instatiation doesn't allow to modify the policy parameters, for instance like this:

policy = LnMlpPolicy(train_env.observation_space ,train_env.action_space, 1, sz_train, sz_train, reuse=False, dueling=False)

But I can try to pass the modified policy to the DQN class instantiation, it doesn't seem to work.
Any advice?

Thanks in advance

[araffin]

Hello,

That's a good question.
For disabling the dueling, you can use net_kwargs argument like that:

from stable_baselines import DQN

model = DQN('MlpPolicy', 'CartPole-v1', policy_kwargs=dict(dueling=False))

For double q learning, we should expose that.
We would appreciate PR that adds that feature ;)