Ml-agents: The Unity environment took too long to respond. ML agents v0.15.1

[alex7e98]

I was trying out the simple RollerBall project with ml agents.
When I run ML agents v0.15.1 & ML agents v0.15.0, there appears to have errors as I type in

mlagents-learn config/config.yaml --run-id=RollerBall-1

(However, it is fine when i switch back to Ml agent v0.14.1)

the erros shows on the cmd window:

Version information:
ml-agents: 0.15.1,
ml-agents-envs: 0.15.1,
Communicator API: 0.15.0,
TensorFlow: 2.0.1
WARNING:tensorflow:From c:\users\alex\python-envs\ml-agents151\lib\site-packages\tensorflow_core\python\compat\v2_compat.py:65: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Process Process-1:
Traceback (most recent call last):
File "C:\Users\Alex\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "C:\Users\Alex\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py", line 99, in run
self._target(self._args, *self._kwargs)
File "c:\users\alex\downloads\ml-agents-0.15.1\ml-agents\mlagents\trainers\subprocess_env_manager.py", line 98, in worker
worker_id, [shared_float_properties, engine_configuration_channel]
File "c:\users\alex\downloads\ml-agents-0.15.1\ml-agents\mlagents\trainers\learn.py", line 444, in create_unity_environment
side_channels=side_channels,
File "c:\users\alex\downloads\ml-agents-0.15.1\ml-agents-envs\mlagents_envs\environment.py", line 143, in __init__
aca_output = self.send_academy_parameters(rl_init_parameters_in)
File "c:\users\alex\downloads\ml-agents-0.15.1\ml-agents-envs\mlagents_envs\environment.py", line 558, in send_academy_parameters
return self.communicator.initialize(inputs)
File "c:\users\alex\downloads\ml-agents-0.15.1\ml-agents-envs\mlagents_envs\rpc_communicator.py", line 98, in initialize
self.poll_for_timeout()
File "c:\users\alex\downloads\ml-agents-0.15.1\ml-agents-envs\mlagents_envs\rpc_communicator.py", line 91, in poll_for_timeout
"The Unity environment took too long to respond. Make sure that :\n"
mlagents_envs.exception.UnityTimeOutException: The Unity environment took too long to respond. Make sure that :
The environment does not need user interaction to launch
The Agents are linked to the appropriate Brains
The environment and the Python interface have compatible versions.
2020-04-09 16:16:09 WARNING [learn.py:343] Unable to save to ./summaries/RollerBall-3_timers.json. Make sure the directory exists

I have checked many posts and I just didnt know how to solve this problem.

Things I already tried to resolve the problem:

1. I have ran the code below before I start training but no luck.
   pip3 install -e ./ml-agents-envs
pip3 install -e ./ml-agents

2. Turning off my firewall, no luck.

The only thing that works is that I switch it back to ml agent V0.14.1

Could anyone please look it up for me, thank you so much.

Screenshots
N/A

Environment (please complete the following information):

• OS + version: Windows 10
• _ML-Agents version_: ML-Agents v0.15.1 & v0.15.0
• _TensorFlow version_: 2.0.1
• _Environment_: RollerBall (https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Create-New.md)

[surfnerd]

Hi @alex7e98,
Did you run your Unity Environment after running the Pyton command in the terminal prompt? We had some users on the forums run into the same issue because the Unity Logo didn't pop up telling them to press play in the Editor. Please let us know if this is your issue.

[alex7e98]

Hi @surfnerd
Thank you for your response.
I have tried to run python command with Unity Enviroment and also without Unity Enviroment. The same issue popped up. Anyway, the Unity Logo did show up in the terminal prompt and even if
I ignore the message and press Play in the Unity Editor, nothing happened.

FYI, I don't know if it is relevant, in Unity Console, the message below keeps showing up.

Couldn't connect to trainer on port 5004 using API version 0.15.0. Will perform inference instead.

However, I can still run ML-Agents v 0.14.1 without any problem but it is not the same case for ML-agent v 0.15.1 & v 0.15.0

I also tried to update Unity version from 2019.3.7f1 to version 2019.3.9f1 but still no luck.

[Zju-George]

Hi @alex7e98,
Did you run your Unity Environment after running the Pyton command in the terminal prompt? We had some users on the forums run into the same issue because the Unity Logo didn't pop up telling them to press play in the Editor. Please let us know if this is your issue.

Same issue here, the Unity Logo did show up but it seems broken in Windows CMD.:sweat_smile:

And the message “press play in the Editor” that supposed to be poped up didn't show up.

But as you suggested, I tried press play after I ran command mlagents-learn config/trainer_config.yaml --run-id=firstRun (actually wait a second). Then the training process starts..:satisfied:

So it seems like it's version 0.15.1's bug. I will continue testing, thanks anyway!

---

(Updated)

Hummmmmm.. Actually it's not training at all!:dizzy_face: Compared to the output log in the document

INFO:mlagents_envs:
'Ball3DAcademy' started successfully!
Unity Academy name: Ball3DAcademy

INFO:mlagents_envs:Connected new brain:
Unity brain name: 3DBallLearning
        Number of Visual Observations (per agent): 0
        Vector Observation space size (per agent): 8
        Number of stacked Vector Observation: 1
        Vector Action space type: continuous
        Vector Action space size (per agent): [2]
        Vector Action descriptions: ,
INFO:mlagents_envs:Hyperparameters for the PPO Trainer of brain 3DBallLearning:
        batch_size:          64
        beta:                0.001
        buffer_size:         12000
        epsilon:             0.2
        gamma:               0.995
        hidden_units:        128
        lambd:               0.99
        learning_rate:       0.0003
        max_steps:           5.0e4
        normalize:           True
        num_epoch:           3
        num_layers:          2
        time_horizon:        1000
        sequence_length:     64
        summary_freq:        1000
        use_recurrent:       False
        summary_path:        ./summaries/first-run-0
        memory_size:         256
        use_curiosity:       False
        curiosity_strength:  0.01
        curiosity_enc_size:  128
        model_path: ./models/first-run-0/3DBallLearning
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 1000. Mean Reward: 1.242. Std of Reward: 0.746. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 2000. Mean Reward: 1.319. Std of Reward: 0.693. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 3000. Mean Reward: 1.804. Std of Reward: 1.056. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 4000. Mean Reward: 2.151. Std of Reward: 1.432. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 5000. Mean Reward: 3.175. Std of Reward: 2.250. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 6000. Mean Reward: 4.898. Std of Reward: 4.019. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 7000. Mean Reward: 6.716. Std of Reward: 5.125. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 8000. Mean Reward: 12.124. Std of Reward: 11.929. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 9000. Mean Reward: 18.151. Std of Reward: 16.871. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 10000. Mean Reward: 27.284. Std of Reward: 28.667. Training.

What I got is

 Version information:
  ml-agents: 0.15.1,
  ml-agents-envs: 0.15.1,
  Communicator API: 0.15.0,
  TensorFlow: 2.0.1
WARNING:tensorflow:From d:\unityprojects\python-envs\sample-env\lib\site-packages\tensorflow_core\python\compat\v2_compat.py:65: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
2020-04-11 23:24:03 INFO [trainer_controller.py:167] Hyperparameters for the PPOTrainer of brain 3DBall:
        trainer:        ppo
        batch_size:     64
        beta:   0.001
        buffer_size:    12000
        epsilon:        0.2
        hidden_units:   128
        lambd:  0.99
        learning_rate:  0.0003
        learning_rate_schedule: linear
        max_steps:      5.0e5
        memory_size:    128
        normalize:      True
        num_epoch:      3
        num_layers:     2
        time_horizon:   1000
        sequence_length:        64
        summary_freq:   12000
        use_recurrent:  False
        vis_encode_type:        simple
        reward_signals:
          extrinsic:
            strength:   1.0
            gamma:      0.99
        summary_path:   firstRun_3DBall
        model_path:     ./models/firstRun/3DBall
        keep_checkpoints:       5
2020-04-11 23:24:03.537975: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-04-11 23:25:27 INFO [trainer.py:214] firstRun: 3DBall: Step: 12000. Time Elapsed: 84.394 s Mean Reward: 1.196. Std of Reward: 0.699. Not Training.
2020-04-11 23:26:48 INFO [trainer.py:214] firstRun: 3DBall: Step: 24000. Time Elapsed: 165.058 s Mean Reward: 1.146. Std of Reward: 0.695. Not Training.
2020-04-11 23:28:08 INFO [trainer.py:214] firstRun: 3DBall: Step: 36000. Time Elapsed: 244.525 s Mean Reward: 1.237. Std of Reward: 0.754. Not Training.
2020-04-11 23:29:28 INFO [trainer.py:214] firstRun: 3DBall: Step: 48000. Time Elapsed: 324.852 s Mean Reward: 1.214. Std of Reward: 0.742. Not Training.

I will try download 0.14! :cry:

[Zju-George]

@surfnerd Thanks God in version 0.14.1... It works...