Baselines: LazyFrame in Atari wrappers
I notice in the comment here: "you won't believe how complex the previous solution was".
I don't understand why it is complicated. The LazyFrame can be implemented simply as an object that holds a list of last _n_ frames. During training, we can call a method to concatenate the list into a numpy array. Because the list stores references, not copies, no memory is wasted. Overloading __array__ seems nothing more than a syntactic sugar that enables np.array(lazy_frame) syntax.
Or am I missing something? Thanks!
miriaford
All 2 comments
This LazyFrame implementation just blew my mind, I think the comment refers about how complex the ReplayBuffer implementation was before LazyFrames. Btw, could you explain how it works? It doesn't make much sense to me on how LazyFrames makes ReplayBuffer store frames only once.
lgvaz
on 30 Aug 2017
could you explain how it works?
The key is to notice that the ReplayBuffer gets whatever observations came out of env.step. When using LazyFrames, this is just a list of references instead of whole numpy arrays. The actual arrays are stored in the heap and the replay buffer is filled with lists of pointers.
Thus, when you ask the replay buffer for a sample, it actually returns a collection of LazyFrames!
nhynes
on 18 Dec 2017
Related issues
louchenyao
路
5Comments
alapite
路
4Comments
miriaford
路
5Comments
Guilherme-B
路
4Comments
damienlancry
路
3Comments
Most helpful comment
The key is to notice that the ReplayBuffer gets whatever observations came out of
env.step. When using LazyFrames, this is just a list of references instead of whole numpy arrays. The actual arrays are stored in the heap and the replay buffer is filled with lists of pointers.Thus, when you ask the replay buffer for a sample, it actually returns a collection of LazyFrames!