Gym: inline support for ipython notebooks

[ryantuck]

it would be awesome to see inline support for ipython notebooks, similar to how %matplotlib inline prevents the secondary python application windows from opening. it seems like an ideal use case for quickly iterating and sharing openai environments (although it is very easy to set up).

i've been unable to find any demos online of people using %pyglet inline, which is discouraging. would this need to be built out at the pyglet level?

[jonasschneider]

That sounds very exciting. I don't think we necessarily need pyglet support; the rgb_array rendering mode might be interesting here -- it just returns the raw image, which we usually use for video encoding. As a first idea, you might want to try using env.render(mode='rgb_array'), and then render the resulting np.ndarray using e.g. matplotlib inside of IPython.

We're definitely open to merging something that makes this even easier, though!

[joschu]

Should work to do plt.imshow(env.render(mode='rgb_array')) inside of your notebook cell

[gdb]

(Going to close for now; let us know if we can help with anything!)

[patrickmineault]

Here's one approach:

http://nbviewer.jupyter.org/github/patrickmineault/xcorr-notebooks/blob/master/Render%20OpenAI%20gym%20as%20GIF.ipynb

[mpacer]

Note import matplotlib.pyplot plt.imshow(env.render(mode='rgb_array')) does not work in either notebook 5.0.0 or 5.1.0. I honestly don't think it would have worked in earlier editions, but some stuff with mimetypes have changed so I don't know if that's affecting it.

But should this have ever worked?:

```python
from matplotlib import pyplot as plt

plt.ion()

%matplotlib inline
import gym
env = gym.make('CartPole-v0')
env.reset()
for _ in range(1000):
plt.imshow(env.render(mode='rgb_array'))
env.step(env.action_space.sample()) # take a random action
```

[mpacer]

I can vouch for the solution by @patrickmineault as working, but I'm thinking this should be possible by using the IPython display mech for handling general mimetypes so long as the right spot can be hooked into directly.

[andrewschreiber]

Wrote up a solution on StackOverflow:

https://stackoverflow.com/questions/40195740/how-to-run-openai-gym-render-over-a-server/45179251#45179251

[dsblank]

I avoided the matplotlib issues by simply doing:

import PIL
PIL.Image.fromarray(env.render(mode='rgb_array')).resize((320, 420))

There may still be an issue on servers without DISPLAYS, but that seems like a separate issue.