Stable-baselines: [Feature discussion] Improving model save format
Continuation on the discussion in #312 related to improving save file format.
Issue with current way of storing/loading models
- Cloudpickle can have issues with different version of Python (mentioned here)
- Cloudpickle itself mentions that it shouldn't be used for long-term storage (something RL people might want).
- Since everything is stored into one serialized file, a single corrupted part prevents easily loading rest of the things. Example: My saved model had Tensorflow code that did not work on different Python version, thus resulting error upon loading. I only needed the model parameters, but I couldn't load them because one of the class items broke deserialization.
- A minor point, but cloudpickle is Python-specific, so accessing anything inside these files requires going through Python+ cloudpickle.
Suggestion
Using Python-native Zipfile to manually construct a zipped archive. Each entry in the zip file would be a separately serialized object. E.g. A2C would have one entry for "gamma", another for "n_steps" and so on. Finally the zip file would include one more entry for the model parameters, serialized with numpy's savez or just with pickle as is done now.
Since many learning algorithms store policies, obs/action spaces, functions and other custom Python objects, it might make sense to store these using pickle. However now if these become non-deserializable (e.g. new Python version), we can still load other items. This works especially well with load_parameters, which could directly access the parameters in zip file rather than trying to deserialize the whole file.
An alternative would be to use JSON to serialize class parameters, but this may not play well with custom functions and the likes. JSON would offer very human-readable and long-lasting files, though. I also took a look at jsonpickle, but I am not sure if relying on another library is a better idea than sticking with pickle for serializing things.
Miffyli
All 10 comments
Hello,
I will be on holidays soon, so I'll let @AdamGleave @erniejunior @hill-a discuss this issue (if they have time), otherwise, I'll try to come by to you at the end of august.
araffin
on 1 Aug 2019
I am a big fan of storing the class parameters in json files. Its easy to parse from other context and will last long enough for all practical purposes.
Not so big fan of the one file per parameter within a zip approach. It seems a bit convoluted to me.
erniejunior
on 5 Aug 2019
I'm back ;)
after encountering some issues due to different package version, i'm definitely for that feature.
I would favor yaml over json (easier to write and it can handle some python objects (cf rl zoo)) and maybe use zip format to have only one file ar the end? (containing npy arrays and serialized params)
The main issue i see is ensuring that we can serialize all python objects.
Last thing: i would keep the cloudpickle format as an option for backward compat at first.
araffin
on 23 Aug 2019
@araffin
Welcome back, hopefully all refreshed! ^^
JSON vs YAML: YAML indeed is more human-readable, but I would go with JSON as it seems more commonly supported (and still seems to require 3rd party libraries for Python). With JSON files, somebody could parse them in other languages/scripts easier. We can use the indent argument of json.dump to make it prettier for human consumption.
Zipping: I agree with @erniejunior "one file per parameter" approach is quite convoluted. How about .zip archive with one JSON file and one file with the parameters (e.g. numpy file)?
Unserializable objects: I figure still will rely on cloudpickle/pickle. How about serializing them into bytes/base64 and storing in the JSON file much like other parameters? This would be tested with json.dump to see which objects are serializable, and if not then use cloudpickle. This will result in messier JSON file, but still readable by other languages.
Backward compatibility: Yes, this would definitely be implemented for loading and as optional setting for saving (just in case).
Miffyli
on 26 Aug 2019
still seems to require 3rd party libraries for Python
Ok, good point.
We can use the indent argument of json.dump to make it prettier for human consumption.
yes, good idea.
How about .zip archive with one JSON file and one file with the parameters (e.g. numpy file)?
Yes, that was my idea + maybe pickle file for what cannot be properly put into one or the other
How about serializing them into bytes/base64 and storing in the JSON file much like other parameters?
This would prevent from using/loading it from another language no?
Btw I think the first step would be to reference/identify what cannot be put into a JSON/numpy file and figure out how to change that, no?
araffin
on 27 Aug 2019
If we encode the bytes of non-JSON-able objects into a string and save it in the JSON, then other languages can still read the JSON file all nice but can't really do much with these byte-strings (they do not have any meaning outside Python/Cloudpickle/Pickle).
Did a quick run of what can be JSONiable and what needs pickling. As expected, spaces and policy are the main issue as they contain classes and functions. However, in my opinion, these can stay byte-serialized as they only have proper meaning in Python. These are also the parts that will likely break something when changing Python or library versions.
One thing we can do with this much like Keras does: When loading model, we can replace some of the items from a file with something else. E.g. A2C.load(..., custom_objects={'policy': other_policy}) would skip reading policy from the file and instead uses the provided one other_policy. This is already in load function but I would move it bit deeper so that it could prevent loading invalid objects (and thus crashing).
<class 'stable_baselines.a2c.a2c.A2C'>
gamma <class 'float'> JSON
n_steps <class 'int'> JSON
vf_coef <class 'float'> JSON
ent_coef <class 'float'> JSON
max_grad_norm <class 'float'> JSON
learning_rate <class 'float'> JSON
alpha <class 'float'> JSON
epsilon <class 'float'> JSON
lr_schedule <class 'str'> JSON
verbose <class 'int'> JSON
policy <class 'abc.ABCMeta'> Cloudpickle
observation_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
action_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
n_envs <class 'int'> JSON
_vectorize_action <class 'bool'> JSON
policy_kwargs <class 'dict'> JSON
<class 'stable_baselines.acer.acer_simple.ACER'>
gamma <class 'float'> JSON
n_steps <class 'int'> JSON
q_coef <class 'float'> JSON
ent_coef <class 'float'> JSON
max_grad_norm <class 'int'> JSON
learning_rate <class 'float'> JSON
lr_schedule <class 'str'> JSON
rprop_alpha <class 'float'> JSON
rprop_epsilon <class 'float'> JSON
replay_ratio <class 'int'> JSON
replay_start <class 'int'> JSON
verbose <class 'int'> JSON
policy <class 'abc.ABCMeta'> Cloudpickle
observation_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
action_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
n_envs <class 'int'> JSON
_vectorize_action <class 'bool'> JSON
policy_kwargs <class 'dict'> JSON
<class 'stable_baselines.acktr.acktr_disc.ACKTR'>
gamma <class 'float'> JSON
nprocs <class 'int'> JSON
n_steps <class 'int'> JSON
vf_coef <class 'float'> JSON
ent_coef <class 'float'> JSON
vf_fisher_coef <class 'float'> JSON
max_grad_norm <class 'float'> JSON
learning_rate <class 'float'> JSON
kfac_clip <class 'float'> JSON
lr_schedule <class 'str'> JSON
verbose <class 'int'> JSON
policy <class 'abc.ABCMeta'> Cloudpickle
observation_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
action_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
n_envs <class 'int'> JSON
_vectorize_action <class 'bool'> JSON
policy_kwargs <class 'dict'> JSON
<class 'stable_baselines.deepq.dqn.DQN'>
checkpoint_path <class 'NoneType'> JSON
param_noise <class 'bool'> JSON
learning_starts <class 'int'> JSON
train_freq <class 'int'> JSON
prioritized_replay <class 'bool'> JSON
prioritized_replay_eps <class 'float'> JSON
batch_size <class 'int'> JSON
target_network_update_freq <class 'int'> JSON
checkpoint_freq <class 'int'> JSON
prioritized_replay_alpha <class 'float'> JSON
prioritized_replay_beta0 <class 'float'> JSON
prioritized_replay_beta_iters <class 'NoneType'> JSON
exploration_final_eps <class 'float'> JSON
exploration_fraction <class 'float'> JSON
learning_rate <class 'float'> JSON
gamma <class 'float'> JSON
verbose <class 'int'> JSON
observation_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
action_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
policy <class 'abc.ABCMeta'> Cloudpickle
n_envs <class 'int'> JSON
_vectorize_action <class 'bool'> JSON
policy_kwargs <class 'dict'> JSON
<class 'stable_baselines.ppo2.ppo2.PPO2'>
gamma <class 'float'> JSON
n_steps <class 'int'> JSON
vf_coef <class 'float'> JSON
ent_coef <class 'float'> JSON
max_grad_norm <class 'float'> JSON
learning_rate <class 'float'> JSON
lam <class 'float'> JSON
nminibatches <class 'int'> JSON
noptepochs <class 'int'> JSON
cliprange <class 'float'> JSON
cliprange_vf <class 'float'> JSON
verbose <class 'int'> JSON
policy <class 'abc.ABCMeta'> Cloudpickle
observation_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
action_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
n_envs <class 'int'> JSON
_vectorize_action <class 'bool'> JSON
policy_kwargs <class 'dict'> JSON
<class 'stable_baselines.trpo_mpi.trpo_mpi.TRPO'>
gamma <class 'float'> JSON
timesteps_per_batch <class 'int'> JSON
max_kl <class 'float'> JSON
cg_iters <class 'int'> JSON
lam <class 'float'> JSON
entcoeff <class 'float'> JSON
cg_damping <class 'float'> JSON
vf_stepsize <class 'float'> JSON
vf_iters <class 'int'> JSON
hidden_size_adversary <class 'int'> JSON
adversary_entcoeff <class 'float'> JSON
expert_dataset <class 'NoneType'> JSON
g_step <class 'int'> JSON
d_step <class 'int'> JSON
d_stepsize <class 'float'> JSON
using_gail <class 'bool'> JSON
verbose <class 'int'> JSON
policy <class 'abc.ABCMeta'> Cloudpickle
observation_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
action_space <class 'gym.spaces.discrete.Discrete'> Cloudpickle
n_envs <class 'int'> JSON
_vectorize_action <class 'bool'> JSON
policy_kwargs <class 'dict'> JSON
Great, I agree on both points ;).
@hill-a @erniejunior @adamgleave could you comment on that so we can start the implementation?
araffin
on 28 Aug 2019
Things being human-readable where possible seems useful for debugging. I don't think there's any need to support languages other than Python, though: even if we're careful with the serialization format, it'd be very hard to make use of this outside of Stable Baselines, since the TensorFlow graph would have to be reconstructed exactly. (If this was a desiderata, we should probably save the graph and weights together.)
This is unrelated, but one thing that continually bugs me when saving models: if you're using VecNormalize, you have to separately save the normalization statistics. It might be nice to have functionality to save both a policy and the set of transformations it used?
AdamGleave
on 29 Aug 2019
@AdamGleave
I agree on both points. Hence I would just cloudpickle/pickle the non-JSON-able objects and leave it at that.
One open point is the storing of model parameters: Should this be Numpy savez object or something more universal? I figure savez (and save) format is long-lasting and easy enough for people to use the values somewhere else-
I did quick experiments with storing objects with this new format, and here is the JSON (class parameters) part of the saved model. Objects that are serialized with cloudpickle (:serialized:) also include first-level members of the object so human reader can get some idea what the serialization contains. These are not used in any way when reading the file (only :serialized: is read and deserialized).
Example JSON file of class parameters
{
"gamma": 0.99,
"timesteps_per_batch": 1024,
"max_kl": 0.01,
"cg_iters": 10,
"lam": 0.98,
"entcoeff": 0.0,
"cg_damping": 0.01,
"vf_stepsize": 0.0003,
"vf_iters": 3,
"hidden_size_adversary": 100,
"adversary_entcoeff": 0.001,
"expert_dataset": null,
"g_step": 1,
"d_step": 1,
"d_stepsize": 0.0003,
"using_gail": false,
"verbose": 0,
"policy": {
":type:": "<class 'abc.ABCMeta'>",
":serialized:": "gASVMgAAAAAAAACMIHN0YWJsZV9iYXNlbGluZXMuY29tbW9uLnBvbGljaWVzlIwJTWxwUG9saWN5lJOULg==",
"__module__": "stable_baselines.common.policies",
"__doc__": "\n Policy object that implements actor critic, using a MLP (2 layers of 64)\n\n :param sess: (TensorFlow session) The current TensorFlow session\n :param ob_space: (Gym Space) The observation space of the environment\n :param ac_space: (Gym Space) The action space of the environment\n :param n_env: (int) The number of environments to run\n :param n_steps: (int) The number of steps to run for each environment\n :param n_batch: (int) The number of batch to run (n_envs * n_steps)\n :param reuse: (bool) If the policy is reusable or not\n :param _kwargs: (dict) Extra keyword arguments for the nature CNN feature extraction\n ",
"__init__": "<function MlpPolicy.__init__ at 0x7feb24b73268>",
"__abstractmethods__": "frozenset()",
"_abc_registry": "<_weakrefset.WeakSet object at 0x7feb24c340f0>",
"_abc_cache": "<_weakrefset.WeakSet object at 0x7feb24c34128>",
"_abc_negative_cache": "<_weakrefset.WeakSet object at 0x7feb24c34198>",
"_abc_negative_cache_version": 52
},
"observation_space": {
":type:": "<class 'gym.spaces.discrete.Discrete'>",
":serialized:": "gASVTgsAAAAAAACME2d5bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpRLCowFc2hhcGWUKYwFZHR5cGWUjAVudW1weZSMBWR0eXBllJOUjAJpOJRLAEsBh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMCW5wX3JhbmRvbZSMFG51bXB5LnJhbmRvbS5fcGlja2xllIwSX19yYW5kb21zdGF0ZV9jdG9ylJOUjAdNVDE5OTM3lIWUUpR9lCiMDWJpdF9nZW5lcmF0b3KUaBSMBXN0YXRllH2UKIwDa2V5lIwVbnVtcHkuY29yZS5tdWx0aWFycmF5lIwMX3JlY29uc3RydWN0lJOUaAiMB25kYXJyYXmUk5RLAIWUQwFilIeUUpQoSwFNcAKFlGgKjAJ1NJRLAEsBh5RSlChLA2gOTk5OSv////9K/////0sAdJRiiULACQAAW6DUsAzWsNOFcUrEouywwBF3V4lPOLX1Xz38oPU+Ft9cmSdH+UdKVn9ZtIUnq6RP6Mn1mqpv9eamqbx1OvrRab59kHcR+kFSd8mlTnwjWpsL797ktjG68OFSTa52PM4iqJpJOdzQFK7Ya4YlvS238Xav/nWTBJgNoxEQ82f0pka7olRWNoR54gIVszlaFWckVM0PArDWId+aJ+TWDqHpOkHom217lPxCqiHJvG/jZ3CyOLA5ID7nwjQRQ2x/6FLJnW2hAPwje7eDUmmSxhm3J2OAuzqZPI30/o9P7OiFAN6aM4wb8Mg3fVua1xIxsgI/88AkcSMhpYXgHnxhf6UkXb/c3pwQ6HQhCWrPEaMgonDwGDMazfV2oSaUmhzWxVP27GpNKRX2OryVeJoKAcCvsXRXkrigucUQiESyksdsTE1HQYmxZeAM2PhNL3hG8jwaUZqsIADZk1vUGMQXX5C9ZMUkHg/ESKviQwacFSKZtt/7RF8iSFB/0zFF/JwQ+xtwueDgwWRB2MnS8q/JvkC7uriGXG1qDEfjaNAM751mZy/X4PQv0AH+YE4o5pYplmO/+hwix/xec3whdl4+KAinMUpLGJMGKERO/i174h8YAmhMepSEN6yQPuTszOCwDJOZGdfZ+ap3SYzC6PIG3ELgXf69Ek++j0zvpRkLP1rCLprf76pd9hgSLrX2kaT1jAbUgnJhdJdEhyJy+4JbXSxc9P6hPLiHZeicFHzXSWTE0hJ0H8PuP8RNEEJe+oL9E4xgSJLLNXFhQd63zVkg6v9bxR0X1pzag8kC1UtF6oH9xhBUoAavFA2J6J+PsHzTsZE5YOGw1YhKv8camBtB+InSXLeL56wwXYc/m+D+fWJeL1P8/hHDB5VuqIajBaEHppbTtH3osXyd9ewomecdRmLYaI6GP+Vy2wlu8/AlCp7fcZChr4JOwcuIqfhU+Ij063ALAQWwglAuHzgO/BAL6Wqw36LCkiARi4WbSROuj3CRYLqhk4AizvA4zDEz7cUCjWm4tS73WkDVeN0U4sa3PDY2uNpwF7Nj9jl6+IwSCv2HDqHvPkOcjwORfQMaJx7i990yst8p+naj13pZCQB9semWWGxbi3LY21UVG8lg5lvmizJBHFpCYTBV4AlaHnk9BBXnmnsZftuFtHvDDXAV3nvvuouP2qQhy3VE3dbmH9xazRQRy4dENoB4ydClla5yAzv6G3fxMs1uxOf5Vo+MZrlmWxCUsitH6A0p2qq1ulMixe73W51u/bcb6JquDCgtFF8xIm8ZGzP+jyvQSi0oPCWH+dIKipQlc7thg8fJ3zxQ0GvNAzmXqoOXr5+OkClJnGA+WWGYEl4Jy5y5UKOKtTmv91BZbciXByuGnGEDk4z/aWrdHBRzLrIqRpo8rvu/XRDpSt1Crh/woK4l1CtohhHfgybR0+dZzHoew5m9Dd0slOdkUFB0YJDFAouVDhbaKfmmxZO14ZS3Lt43yBEiv67U5cHwNdudZP7bnQV8bP2JF0a9atXC+h/iHBZgMJSYvByflIEFMkZUAUyeq3mDeudXTj6f6EIOduOuqGLNgZui9Sy303WUpyD+IHZQt+PwbFyae+gaGQ4wFN7zp0B2fnNYwb7K0RQImf4/4gkABgQp/J5h4JVQXxt+f06dJLHroswT2MhUR2b/7go0O/D7GewQaB4SWLmRpzmCFsOHp7akEiRPFLQrOXvg0jgiUUptf0cW+LuyeLo3bYbs9k8bkvOX/VgbdSujE6SvGA6oSe6ztUT11dvw8+aft0IWMa9EnJzILXa1NShJVM6nTu2r2cZRG7vaqQE4OJTr8XQZw8qMgBbStI/NRLfgbSA+XJIVV/L6ixpRYQCC5lHlbHcnSMoJi0pSYZ5fr71eWjDijhULcH6csQnIO/YNZKKJF3Q0xbxaqP6aq7C3RsIRoQsls2sSv9/uZ4IQ4D/ioe8m397X/Zs6QJZ/hYK0nXZn1c6dDSOaDJibVtkKBUTdWhbo6e93y1bwMWC+PYkUkxAmEx2Y1SnntwoUmiJYH179BCQLignL1omY+HagH5ZP5odULYNdU/A4oJHoSEgPxYDtnta4VRsu21fN5a9Tx/+0rA08cghouw32yH4A7qjboJdpX3b7YhKinhhzjFtNLmjy79k6o5KDLM9vb6oswJFAp82SEFkLCiycVvUTTE7pwOB3PpRzcHUdHTBfnPF0FuPp//hjNv7srG5XjzuiH9wvmK1oYxsnvm2vqcomlBE0gjiW/nsqXybwJtqpGxYu8ipbqsYpqDN+QzRtu+S+jRfnwdkcAazG9cpasrAoUDhWTYzVbgNJBdraafYa6WxJNXdlB26/CkgXQEAJ28qZDLccx7h20V+0NH/Mj9fMd2+jOq9Ksd50k5MQqzWosk4dynbUhbGM/atehHVf10fxTWftyXBL1aI+FvaGKJj/NxcxnhWhNwFn5zd0tBgI6BpXZup/vdsMrc5S+ZrRafbQi5Rhhjr88gZHmOcBMljLe7cOLtmXhqk/HrI+47ftamQ8Oy6MGY+F6RcxSOWlQpU4w3MjhfnKqw9NEc18+Bjg9O/+60oj9SNNteMW8xL/d2D6+PS0k8R18oFQl/zOk9yBbgeqYkYTZ39pFygk1KdbbgybFkj/VMwkWEKGy+qIFWP2NWbVHf0nqd/U8rVfvo9PHcDF92HphtkYGqqMgbwfZhQWJfXO8Ijw7f+vvWZzgpt2khJSwYaBiJH9czE2rg5ps4CDlQ64nKnGWE/USQCAfyicP7MesC2+wE1gmLMR4WCDLHq7YhaRHTpUPmYqWx9xuB9AVIIL147OZPeYTfqTHDI53kbVnw4nqds4pAa64MvQtsZT/r46alHXeWJDvIwF3S2E1hHoGsXSt62RPtZpbcZ1hl4wZqaM3tkuX6ZIWsFJ7+Q8pGrPrvWdaFRMg/g/QDE0SmWu706c3q0yPYoEdqxQldbeEveY81V0sR6rRftLnWWGJ637ujgDNYujxwCZJpJ52hy8gdhTgli78HKd9nhoqx2wo3BS403A9CCAcSOCTFR8zCde6foFoPCp0pUp6DAtNLwvpmdNBTRlfF4cLh38FUvfM8r6gwNs3WFQ63pyy9fIHFPI6ApUFcgZ1VbmnA9jUOiJ6KKhQygLFZ/q91ZCqCYTgDfUUj8UqWXfpa4Ucmljc6KX2JDgDhKFa5wvYUiIHO6LdMClSU6sc5FxVpMJOx+jFgD2O7HII9kueQM9QjKjFo7H4ZxX1uX88a+oV04JkJDIDSd3iTFqqfOn//1KrCutmxdLoyHJ0PeizgDmcQbFsrGi0qya1LsW2PdIuwXscMRJ7ZRJlHSUYowDcG9zlEsDdYwJaGFzX2dhdXNzlEsAjAVnYXVzc5RHAAAAAAAAAAB1YnViLg==",
"n": 10,
"shape": [],
"dtype": "int64",
"np_random": "RandomState(MT19937)"
},
"action_space": {
":type:": "<class 'gym.spaces.discrete.Discrete'>",
":serialized:": "gASVTgsAAAAAAACME2d5bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpRLCowFc2hhcGWUKYwFZHR5cGWUjAVudW1weZSMBWR0eXBllJOUjAJpOJRLAEsBh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMCW5wX3JhbmRvbZSMFG51bXB5LnJhbmRvbS5fcGlja2xllIwSX19yYW5kb21zdGF0ZV9jdG9ylJOUjAdNVDE5OTM3lIWUUpR9lCiMDWJpdF9nZW5lcmF0b3KUaBSMBXN0YXRllH2UKIwDa2V5lIwVbnVtcHkuY29yZS5tdWx0aWFycmF5lIwMX3JlY29uc3RydWN0lJOUaAiMB25kYXJyYXmUk5RLAIWUQwFilIeUUpQoSwFNcAKFlGgKjAJ1NJRLAEsBh5RSlChLA2gOTk5OSv////9K/////0sAdJRiiULACQAAW6DUsAzWsNOFcUrEouywwBF3V4lPOLX1Xz38oPU+Ft9cmSdH+UdKVn9ZtIUnq6RP6Mn1mqpv9eamqbx1OvrRab59kHcR+kFSd8mlTnwjWpsL797ktjG68OFSTa52PM4iqJpJOdzQFK7Ya4YlvS238Xav/nWTBJgNoxEQ82f0pka7olRWNoR54gIVszlaFWckVM0PArDWId+aJ+TWDqHpOkHom217lPxCqiHJvG/jZ3CyOLA5ID7nwjQRQ2x/6FLJnW2hAPwje7eDUmmSxhm3J2OAuzqZPI30/o9P7OiFAN6aM4wb8Mg3fVua1xIxsgI/88AkcSMhpYXgHnxhf6UkXb/c3pwQ6HQhCWrPEaMgonDwGDMazfV2oSaUmhzWxVP27GpNKRX2OryVeJoKAcCvsXRXkrigucUQiESyksdsTE1HQYmxZeAM2PhNL3hG8jwaUZqsIADZk1vUGMQXX5C9ZMUkHg/ESKviQwacFSKZtt/7RF8iSFB/0zFF/JwQ+xtwueDgwWRB2MnS8q/JvkC7uriGXG1qDEfjaNAM751mZy/X4PQv0AH+YE4o5pYplmO/+hwix/xec3whdl4+KAinMUpLGJMGKERO/i174h8YAmhMepSEN6yQPuTszOCwDJOZGdfZ+ap3SYzC6PIG3ELgXf69Ek++j0zvpRkLP1rCLprf76pd9hgSLrX2kaT1jAbUgnJhdJdEhyJy+4JbXSxc9P6hPLiHZeicFHzXSWTE0hJ0H8PuP8RNEEJe+oL9E4xgSJLLNXFhQd63zVkg6v9bxR0X1pzag8kC1UtF6oH9xhBUoAavFA2J6J+PsHzTsZE5YOGw1YhKv8camBtB+InSXLeL56wwXYc/m+D+fWJeL1P8/hHDB5VuqIajBaEHppbTtH3osXyd9ewomecdRmLYaI6GP+Vy2wlu8/AlCp7fcZChr4JOwcuIqfhU+Ij063ALAQWwglAuHzgO/BAL6Wqw36LCkiARi4WbSROuj3CRYLqhk4AizvA4zDEz7cUCjWm4tS73WkDVeN0U4sa3PDY2uNpwF7Nj9jl6+IwSCv2HDqHvPkOcjwORfQMaJx7i990yst8p+naj13pZCQB9semWWGxbi3LY21UVG8lg5lvmizJBHFpCYTBV4AlaHnk9BBXnmnsZftuFtHvDDXAV3nvvuouP2qQhy3VE3dbmH9xazRQRy4dENoB4ydClla5yAzv6G3fxMs1uxOf5Vo+MZrlmWxCUsitH6A0p2qq1ulMixe73W51u/bcb6JquDCgtFF8xIm8ZGzP+jyvQSi0oPCWH+dIKipQlc7thg8fJ3zxQ0GvNAzmXqoOXr5+OkClJnGA+WWGYEl4Jy5y5UKOKtTmv91BZbciXByuGnGEDk4z/aWrdHBRzLrIqRpo8rvu/XRDpSt1Crh/woK4l1CtohhHfgybR0+dZzHoew5m9Dd0slOdkUFB0YJDFAouVDhbaKfmmxZO14ZS3Lt43yBEiv67U5cHwNdudZP7bnQV8bP2JF0a9atXC+h/iHBZgMJSYvByflIEFMkZUAUyeq3mDeudXTj6f6EIOduOuqGLNgZui9Sy303WUpyD+IHZQt+PwbFyae+gaGQ4wFN7zp0B2fnNYwb7K0RQImf4/4gkABgQp/J5h4JVQXxt+f06dJLHroswT2MhUR2b/7go0O/D7GewQaB4SWLmRpzmCFsOHp7akEiRPFLQrOXvg0jgiUUptf0cW+LuyeLo3bYbs9k8bkvOX/VgbdSujE6SvGA6oSe6ztUT11dvw8+aft0IWMa9EnJzILXa1NShJVM6nTu2r2cZRG7vaqQE4OJTr8XQZw8qMgBbStI/NRLfgbSA+XJIVV/L6ixpRYQCC5lHlbHcnSMoJi0pSYZ5fr71eWjDijhULcH6csQnIO/YNZKKJF3Q0xbxaqP6aq7C3RsIRoQsls2sSv9/uZ4IQ4D/ioe8m397X/Zs6QJZ/hYK0nXZn1c6dDSOaDJibVtkKBUTdWhbo6e93y1bwMWC+PYkUkxAmEx2Y1SnntwoUmiJYH179BCQLignL1omY+HagH5ZP5odULYNdU/A4oJHoSEgPxYDtnta4VRsu21fN5a9Tx/+0rA08cghouw32yH4A7qjboJdpX3b7YhKinhhzjFtNLmjy79k6o5KDLM9vb6oswJFAp82SEFkLCiycVvUTTE7pwOB3PpRzcHUdHTBfnPF0FuPp//hjNv7srG5XjzuiH9wvmK1oYxsnvm2vqcomlBE0gjiW/nsqXybwJtqpGxYu8ipbqsYpqDN+QzRtu+S+jRfnwdkcAazG9cpasrAoUDhWTYzVbgNJBdraafYa6WxJNXdlB26/CkgXQEAJ28qZDLccx7h20V+0NH/Mj9fMd2+jOq9Ksd50k5MQqzWosk4dynbUhbGM/atehHVf10fxTWftyXBL1aI+FvaGKJj/NxcxnhWhNwFn5zd0tBgI6BpXZup/vdsMrc5S+ZrRafbQi5Rhhjr88gZHmOcBMljLe7cOLtmXhqk/HrI+47ftamQ8Oy6MGY+F6RcxSOWlQpU4w3MjhfnKqw9NEc18+Bjg9O/+60oj9SNNteMW8xL/d2D6+PS0k8R18oFQl/zOk9yBbgeqYkYTZ39pFygk1KdbbgybFkj/VMwkWEKGy+qIFWP2NWbVHf0nqd/U8rVfvo9PHcDF92HphtkYGqqMgbwfZhQWJfXO8Ijw7f+vvWZzgpt2khJSwYaBiJH9czE2rg5ps4CDlQ64nKnGWE/USQCAfyicP7MesC2+wE1gmLMR4WCDLHq7YhaRHTpUPmYqWx9xuB9AVIIL147OZPeYTfqTHDI53kbVnw4nqds4pAa64MvQtsZT/r46alHXeWJDvIwF3S2E1hHoGsXSt62RPtZpbcZ1hl4wZqaM3tkuX6ZIWsFJ7+Q8pGrPrvWdaFRMg/g/QDE0SmWu706c3q0yPYoEdqxQldbeEveY81V0sR6rRftLnWWGJ637ujgDNYujxwCZJpJ52hy8gdhTgli78HKd9nhoqx2wo3BS403A9CCAcSOCTFR8zCde6foFoPCp0pUp6DAtNLwvpmdNBTRlfF4cLh38FUvfM8r6gwNs3WFQ63pyy9fIHFPI6ApUFcgZ1VbmnA9jUOiJ6KKhQygLFZ/q91ZCqCYTgDfUUj8UqWXfpa4Ucmljc6KX2JDgDhKFa5wvYUiIHO6LdMClSU6sc5FxVpMJOx+jFgD2O7HII9kueQM9QjKjFo7H4ZxX1uX88a+oV04JkJDIDSd3iTFqqfOn//1KrCutmxdLoyHJ0PeizgDmcQbFsrGi0qya1LsW2PdIuwXscMRJ7ZRJlHSUYowDcG9zlEsDdYwJaGFzX2dhdXNzlEsAjAVnYXVzc5RHAAAAAAAAAAB1YnViLg==",
"n": 10,
"shape": [],
"dtype": "int64",
"np_random": "RandomState(MT19937)"
},
"n_envs": 1,
"_vectorize_action": true,
"policy_kwargs": {}
}
Should this be Numpy savez object or something more universal?
Using numpy format sounds reasonable.
I did quick experiments with storing objects with this new format, and here is the JSON
Nice ;)
araffin
on 29 Aug 2019
Related issues
stefanbschneider
路
3Comments
matthew-hsr
路
3Comments
saeid93
路
3Comments
shwang
路
3Comments
sahilgupta2105
路
3Comments