Ml-agents: Roadmap Update (Jan 21, 2020)

[unityjeffrey]

Greetings ML-Agents Community,

Happy New Year to everyone!

We are happy to announce that going forward we will be publishing and updating a roadmap with each release. We strongly believe in transparency and ensuring our users are successful. Our initial roadmap will contain a 3-month view on features and work our team (the core Unity ML-Agents team) has committed. We will update this 3-month view after each release. In the coming months, we will find a more permanent home for announcing and tracking the roadmap as well as enabling users to submit feedback and requests. For now, please continue to use Github issues and we will announce a new system.

We also plan to leverage Unity Forum as well. For those not familiar, Unity Forum is where a majority of Unity game developers and users collaborate. We will always continue to leverage Github for bugs and issues with the ML-Agents repository. For now, please continue to use Github Issues while we set up our own space on Unity Forum.

Since this is our first announcement and publishing of our roadmap, I wanted to provide a general announcement for the next few months.

As you may have already noticed, there is a significant amount of refactoring going on in both the C#, communication layer, and Python components of ML-Agents. Our goal in the next three months is to make components much more stable. Specifically -

• Simplified installation, integration, and usage
• Reduced complexity of our codebase
• Improved maintainability of our repo

We are confident that for our researchers, we are creating flexibility in the toolkit for exploration of new methods and algorithms. And for our core Unity developers, that ML-Agents is a reliable solution for implementing deep learning agents in Unity content. We’ll be making more announcements on this in the coming months.

Packages - We will be releasing a preview Unity package for ML-Agents that will be available in the Unity package manager. We hope this will resolve many production and implementation issues with our Unity developers. Our plan is to first release this package for object-oriented (Monobehavior) in March. We plan to release a package for ECS in 2020.

Support and versioning - We’ve received feedback about improving versioning and support for different Unity versions and ML-Agents. With every release of the Github repo or Unity package, we want to ensure our code works with the actively supported versions of Unity (LTS). In the next 3 months, we will be calling out which versions Unity will be officially supported in the repo and package. We hope this will give our Unity developers clarity on version support. More details to come on this.

ML-Agents 3 Month Roadmap as of Jan 21, 2020 -

• Repository restructuring to support packages
  
  
  
  • com.unity.ai.ml-agents
  
  
  • com.unity.ai.ml-agents.examples
  
  
  • com.unity.ai.ml-agents-ecs
  
  
  • com.unity.ai.ml-agents-ecs.examples
  
  
  • mlagents-envs (Low Level Python API)
  
  
  • mlagents-trainers (RL Algorithm development)
  
  
  • Project (required for package testing/verification)
    
    
    
    
    
    • Assets
    
    
    
    • Packages
    
    
    
  
  
  • manifest.json (include packages for testing)
  
  
• Release of ML-Agents Unity preview package (Object-Oriented, Mono-behavior)
• Refactoring of the Unity SDK (C#) codebase
  
  
  
  • Introduction of Reward Providers
  
  
• Refactoring of the Trainers (Python)codebase
  
  
  
  • Support for new LL-Python API, including Multi-Agent/Teams
  
  
  • Redesign and refactor of Trainer, TrainerController, and Buffer interfaces to no longer rely on BrainInfo and have a well-defined/documented interface
  
  
  • Breakdown of Trainer, Policy, and Model into re-usable bite-sized modules, and re-implementation of SAC and PPO using these modules
  
  
• Multi-agent algorithm for self-play
  
  
  
  • Ghost Trainer algorithm
  
  
  • Updated to the soccer and tennis using Ghost Trainer
  
  
• Sensors and observations
  
  
  
  • Support uncompressed (float) visual observations
  
  
• Support for ONNX model export and import
• Updated docker images as part of each ML-Agents release

[caioc2]

As a user since the 0.5, I like what ml-agents propose and I hope for a great tool in the future, but this are my 2 cents on what I think it should improve.

I've seen many new features come to life, various changes to the usage and implementation which sounds good, but many bugs are not being look at or are "low priority" and still open.

IMHO, it would be much more porwefull to offer a simple tool which may lack some fancy stuff but is very reliable. In general, every Unity package seems to be always in preview and never reach the quality of a release.

Just to cite some of the things that have been a problem and I think could be handled better:

• Versions (which may seems to be in focus now) - Having to update to a new version to solve old bugs and refactor all your code, to discover new bugs. Also having completely different models from different versions, without a solid explanation why and which changes are related to it.
• IL2CPP compiler
• Barracuda plagued with bugs. I think the idea to have a universal inference engine is amazing, but if you are not willing to invest the necessary amount of resources to build such tool with great quality, then it's best to use something else that may not fit perfectly your purpose but is well tested and documented.
• Fast paced development of new things while slow pace to fix or improve core features.

TL;DR - stabillity and heavily tested core functionality is a must, one can't build and test new algorithms when the core framework cant be trusted.

Anyway, I'm looking forward it, transparency is always a good sign.

[unityjeffrey]

thank you for your comment @caioc2. we also have this mindset of trying to not introduce too many new features but try and make the core codebase more stable. As we improve our version of Unity support, we are also striving to ensure bug fixes are applied to previous versions so you are not having to update new versions just to get the fix. I've also passed some of your feedback to the Barracuda team here at Unity as well.

Again, we really appreciate feedback.

[lostmsu]

@unityjeffrey I see you mentioned Unity SDK (C#) codebase. Is there a NuGet package, that can be used to interact with Unity-based environment for ML agents similar to mlagents on pip?

[unityjeffrey]

hi @lostmsu - i don't think we have this planned, however @surfnerd may have some more insights on it. Could you describe what you would be looking for in particular?