Gym: Creating gym environments

[DecentGradient]

Would it be possible to create more in depth documentation on creating new gym environments?

[mariomeissner]

I agree that there should be more documentation on basically everything here, especially on environments.

[cshr25]

I am not quite sure how to interact with each environment here... still learning. BTW why can't I see some of the algorithm code such as this one https://gym.openai.com/evaluations/eval_27mYElciS7aZ7AkjP8RZQ ? I can only see its performance.

[olegklimov]

Env interface is dead easy. gym.make(), reset(), step(). Sometimes you want render(). That's it. Easy from both sides, using and implementing.

To make a good environment, you basically have to have game programming skills. Not something we can cover in docs here.

[DecentGradient]

What about Space?

On Mon, Jul 17, 2017, 2:13 PM Oleg Klimov [e-mail] wrote:

Env interface is dead easy. gym.make(), reset(), step(). Sometimes you
want render(). That's it. Easy from both sides, using and implementing.

To make a good environment, you basically have to have game programming
skills. Not something we can cover in docs here.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/openai/gym/issues/610#issuecomment-315836320, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEZlcEDP1LhUPppnYvcMnuv3s_GD7j-sks5sO6RLgaJpZM4NvoDH
.

[olegklimov]

Right, also you have to read Box, Discrete and Tuple which is one screen of code more.

[DecentGradient]

It's been a month since I tried, but at the time it was I concluded that it was not worth it to spend the time digging through comments in the various files to understand how gym implements action and state space. I reimplemented these parts on my own. I will take another look at it and see if it is clearer to me now.

While something may seem obvious to you, as part of the project, it may not be as readily apparent to a another person's despite many years of programming experience.

Certainly you can see that having separate documentation, as opposed to (very well ) commented source code would make openai's projects more accessible. With effort, even "dead easy" subjects can be explained.

I hope that you and your team recognize the benefit of improving the documentation for this project. Keep up the good work:)

[jaredthecoder]

I also agree. Even for the toy text examples, I've been trying to create my own environment based off the CliffWalker environment, and the extreme lack of comments in Discrete.py and others makes things more difficult than necessary.

If you want people to add more value to this project in the form of environments, which ultimately allow testing of ML algorithms, you need actual documentation.

[MrEmanuel]

In the spirit of standardization I think that the ML community would benefit from documentation that makes it straightforward to create your own environments. Like it says on the openAI website;

RL research is also slowed down by...
Lack of standardization of environments used in publications. Subtle differences in the problem definition, such as the reward function or the set of actions, can drastically alter a task’s difficulty. This issue makes it difficult to reproduce published research and compare results from different papers.

Applying a few high-level functions and models on a pre-made environments is all good and well when just starting out but as soon as you want to apply these functions on your own system or problem, you're missing that interface that came pre-packed for you with environments.

With better guidance on how to create your own environment, you could more easily modell any arbitrary problem and apply the abstract tools available through for example tflearn in a standardized way.
I'm working on a simulation of a mechanical system myself and started looking at how to building my own environment from the gym source code but I found it hard to get a grip on it. Granted I am not a senior developer or have much experience with ML but I think that's exactly why better documentation is needed. :)

Otherwise, awesome resources.

[DecentGradient]

Very well said.

On Sun, Mar 11, 2018, 2:37 PM MrEmanuel [e-mail] wrote:

In the spirit of standardization I think that the ML community would
benefit from documentation that makes it straight-forward to create your
own environments. Like it says on the openAI website;

Lack of standardization of environments used in publications. Subtle
differences in the problem definition, such as the reward function or the
set of actions, can drastically alter a task’s difficulty. This issue makes
it difficult to reproduce published research and compare results from
different papers.

Applying a few high-level functions and models on a pre-made environments
is all good and well when just starting out but as soon as you want to
apply these functions on your own system or problem, you're missing that
interface that came pre-packed for you with environments.

With better guidance on how to create your own environment, you could more
easily modell any arbitrary problem and apply the abstract tools available
through for example tflearn in a standardized way.
I'm working on a simulation of a mechanical system myself and started
looking in to building my own environment from the gym source code but I
found it hard to get a grip on it. Granted I am not a senior developer or
have much experience with ML but I think that's exactly why better
documentation is needed. :)

Otherwise, awesome resources.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/openai/gym/issues/610#issuecomment-372138010, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEZlcAv24sROHGfGg_HuoKmc8cYrVBraks5tdW78gaJpZM4NvoDH
.

[jperl]

This is a good initial resource in case it is helpful. How to create new environments for Gym.

And I have been using Ray's custom env as an updated example (the soccer one is out of date).

[christopherhesse]

The current guide is here: https://github.com/openai/gym/blob/master/docs/creating-environments.md

If anyone would like to improve those docs, we would welcome PRs. As @jperl mentioned the soccer env is out of date so an updated one would be nice to have.

In addition, we could export docstrings into a readable doc for the relevant classes (gym.Env, gym.spaces.*) like retro does here: https://retro.readthedocs.io/en/latest/python.html

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.