Stable-baselines: Using callback for GAIL [question]

[prabhasak]

I am trying to save the GAIL model during training. I'm using the example code from the Callback documentation as reference:

from stable_baselines import GAIL
from stable_baselines.gail import ExpertDataset, generate_expert_traj
from stable_baselines.common.callbacks import CheckpointCallback

checkpoint_callback = CheckpointCallback(save_freq=1000, save_path='./logs/', name_prefix='gail')
dataset = ExpertDataset(expert_path='C:/IL_UAV/GAIL-TF-SB/stable_baselines/gail/dataset/expert_pendulum.npz', traj_limitation=10, verbose=1)

model = GAIL('MlpPolicy', 'Pendulum-v0', dataset, verbose=1)
model.learn(2000, callback=checkpoint_callback)

I get the following **AttributeError:** 'NoneType' object has no attribute 'process'. What should I be doing?

PS: I am relatively new to both Stable Baselines and RL!

System Info

• Python 3.7.0
• Tensorflow 1.14.0

[araffin]

Hello,
Can you give the full traceback ?
And how did you record the expert data?

[prabhasak]

Oops, apologies. Here's the traceback (for the code mentioned earlier) :

Traceback (most recent call last):
  File "checking_callback.py", line 9, in <module>
    model.learn(2000, callback=checkpoint_callback)
  File "C:\IL_UAV\GAIL-TF-SB\stable_baselines\gail\model.py", line 54, in learn
    return super().learn(total_timesteps, callback, log_interval, tb_log_name, reset_num_timesteps)
  File "C:\IL_UAV\GAIL-TF-SB\stable_baselines\trpo_mpi\trpo_mpi.py", line 456, in learn
    ob_expert, ac_expert = self.expert_dataset.get_next_batch()
  File "C:\IL_UAV\GAIL-TF-SB\stable_baselines\gail\dataset\dataset.py", line 152, in get_next_batch
    if dataloader.process is None:
AttributeError: 'NoneType' object has no attribute 'process'

The expert data for Inverted Pendulum is available in stable_baselines/gail/dataset, so I did not have to record it. Thanks!

[prabhasak]

Hi. The issue is still unresolved. Closed this by mistake! (opened another better-worded issue)

[haochihlin]

Hi @prabhasak
I faced the same error, could you please give me more detail that how you fix it?
As you mentioned by adding 'if__name__=='__main__':' can avoid the error, I was wondering if you are suggesting as follows?

if __name__=='__main__':
    checkpoint_callback = CheckpointCallback(save_freq=1000, save_path='./logs/', name_prefix='gail')
    dataset = ExpertDataset(expert_path='C:/IL_UAV/GAIL-TF-SB/stable_baselines/gail/dataset/expert_pendulum.npz', traj_limitation=10, verbose=1)

    model = GAIL('MlpPolicy', 'Pendulum-v0', dataset, verbose=1)
    model.learn(2000, callback=checkpoint_callback)

[prabhasak]

Hey @haochihlin!

You're right, sincere apologies (just edited my earlier reply.) I checked things in a hurry, and wrote just main in place of __main__ so the code never really ran, and hence showed no errors. There are a quite a few mistakes and inefficiencies in this whole thread.

@araffin, it would be great if you could delete this issue. I will open a new issue with a clear description of the error. Sorry for the trouble!

[haochihlin]

Hi @prabhasak
Thanks for the reply.
I tried to debug and trace the code, I think the problem comes from the following line:
https://github.com/hill-a/stable-baselines/blob/master/stable_baselines/trpo_mpi/trpo_mpi.py#L455

When enabling the callback func, the 'self.expert_dataset' is somehow becoming new DataLoader obj without initialization ...