Ray: Known issues on Windows

[mehrdadn]

This page is intended to list known Ray issues on Windows in one central location.

The latest nightly wheels may have already addressed some issues since the latest official release.
Check back here for updates as issues are addressed.

You can vote by reacting 👍 on each issue that is impacting you to help us prioritize issues. 🙂

(Maintainers: Please reference this issue in other posts. That will allow their statuses to show up here.)

• [TODO] Ray Serve, Ray SGD, autoscaler, multi-node usage, etc. may not work
• [TODO] Dashboard doesn't work on Windows (related to Node incompatibilities)
• #9265: Unable to run multiple instances of Ray at once
• #9259: Multi-node connection failure to Redis
• #9239: AttributeError: 'AsyncStream' object has no attribute 'fileno'
• #9196: assert num_imports >= num_imported with PyTorch and Ray Tune
• ~#9166: GPU not recognized by Ray Tune~
• ~#9128: FileExistsError with Ray Tune~
• #9117: Missing Unicode support in C++ core
• #9116: Check failed: assigned_port != -1 on virtual Python 3.7 or 3.8 environments
• #9083: ImportThread: max number of clients reached and Check failed: _s.ok() Bad status: RedisError
• #9074: Access violation in msvcp140.dll!mtx_do_lock, called from RedisAsyncContext::RedisAsyncHandleRead
• #8787: assertion failed: grpc_server_request_registered_call(...) == GRPC_CALL_OK
• wookayin/gpustat#57: gpustat currently requires pip install git+https://github.com/wookayin/gpustat

[cristiangofiar]

FileNotFoundError: [Errno 2] Dashboard build directory not found. If installing from source, please follow the additional steps required to build the dashboard(cd python/ray/dashboard/client && npm ci && npm run build): 'C:\Users\crist\anaconda3\lib\site-packages\ray\dashboard\client/build'

Help!

[richardliaw]

@cristiangofiar thanks for opening this issue! That should be non-fatal; we should reduce the severity of that error.

cc @mfitton maybe let's just log an "info" message rather than Error or Warning.

[cristiangofiar]

@richardliaw But this failure can affect the execution of the program? I need to use Ray for an integrative job on a college subject!
Also, the bug bothers!
You can help?

[valentasgruzauskas]

Runing exeriment with HyperOptSearch and LightGBM, and receive rror message.

raise TuneError("Trials did not complete", incomplete_trials)

OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\Users\User\ray_results\train_flat_price\train_flat_price_1_bagging_fraction=tune.sample_from(. at 0x000001C05ECACE58>),feature_fraction=_2020-08-20_18-00-270k00linp'

[mfitton]

@cristiangofiar As a short term fix you could disable the dashboard by using the argument --include-webui=False at the command line or include_webui=False in the call to ray.init() in your python code depending how you start it up. (Note this argument is being changed to --include-dashboard and include_dashboard respectively, but I don't know what version you're using.)

There are other issues with the Dashboard on Windows still that are still being fixed. Currently, even if you get the dashboard to start, it won't render anything. That said, this will not affect the running of your script.

[cristiangofiar]

@mfitton Thanks you very much!
I have fixed it thanks to you! :D

[richardliaw]

OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\Users\User\ray_results\train_flat_price\train_flat_price_1_bagging_fraction=tune.sample_from(),feature_fraction=_2020-08-20_18-00-270k00linp'

@valentasgruzauskas can you post a longer stacktrace?

[valentasgruzauskas]

The fatal error was a problem from my side, I used tune to generate input data, but used a search algorithm. Now I define the search space with hyperopt randint, uniform etc. and it works (at least no fatal errors). However, I keep receiving an error.

2020-08-22 13:47:05,381 WARNING util.py:137 -- The experiment_checkpoint operation took 10.414000749588013 seconds to complete, which may be a performance bottleneck.
2020-08-22 13:47:05,382 ERROR trial_runner.py:375 -- Trial Runner checkpointing failed.
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\envs\ntanalysis\lib\site-packages\ray\tune\trial_runner.py", line 373, in step
self.checkpoint()
File "C:\ProgramData\Anaconda3\envs\ntanalysis\lib\site-packages\ray\tune\trial_runner.py", line 302, in checkpoint
self._local_checkpoint_dir, session_str=self._session_str)
File "C:\ProgramData\Anaconda3\envs\ntanalysis\lib\site-packages\ray\tune\suggest\search_generator.py", line 192, in save_to_dir
base_searcher.save_to_dir(dirpath, session_str)
File "C:\ProgramData\Anaconda3\envs\ntanalysis\lib\site-packages\ray\tune\suggest\suggestion.py", line 210, in save_to_dir
self.CKPT_FILE_TMPL.format(session_str)))
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\Users\User\ray_results\train_flat_price\.tmp_searcher_ckpt' -> 'C:\Users\User\ray_results\train_flat_price\searcher-state-2020-08-22_12-24-21.pkl'

[cristiangofiar]

Help to connect 2 PCs pls!

(base) C:\Users\Gofiar>ray start --address='address' --redis-password='pass'
Traceback (most recent call last):
File "c:\users\gofiar\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"_main_", mod_spec)
File "c:\users\gofiar\anaconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\Gofiar\anaconda3\Scripts\ray.exe_main_.py", line 7, in
File "c:\users\gofiar\anaconda3\lib\site-packages\ray\scripts\scripts.py", line 1237, in main
return cli()
File "c:\users\gofiar\anaconda3\lib\site-packages\click\core.py", line 764, in _call_
return self.main(args, *kwargs)
File "c:\users\gofiar\anaconda3\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "c:\users\gofiar\anaconda3\lib\site-packages\click\core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\gofiar\anaconda3\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, *ctx.params)
File "c:\users\gofiar\anaconda3\lib\site-packages\click\core.py", line 555, in invoke
return callback(args, **kwargs)
File "c:\users\gofiar\anaconda3\lib\site-packages\ray\scripts\scripts.py", line 399, in start
address, redis_address)
File "c:\users\gofiar\anaconda3\lib\site-packages\ray\services.py", line 279, in validate_redis_address
redis_address = address_to_ip(redis_address)
File "c:\users\gofiar\anaconda3\lib\site-packages\ray\services.py", line 311, in address_to_ip
ip_address = socket.gethostbyname(address_parts[0])
socket.gaierror: [Errno 11001] getaddrinfo failed

[Dris101]

I had a similar issue: After ray start --head the head node prints:-
--------------------
Ray runtime started.
--------------------

Next steps
To connect to this Ray runtime from another node, run
ray start --address='192.168.143.221:6379' --redis-password='password here'

but address_to_ip(address): in services.py does not trim the quotes from the IP address so socket.gethostbyname(address_parts[0]) throws an error. The message from ray start is misleading. Try without the quotes around the IP address.

[talhaanwarch]

FileNotFoundError: [Errno 2] Dashboard build directory not found. If installing from source, please follow the additional steps required to build the dashboard(cd python/ray/dashboard/client && npm ci && npm run build): 'C:\Users\crist\anaconda3\lib\site-packages\ray\dashboard\client/build'

Help!

any solution for this

[mfitton]

@talhaanwarch the dashboard currently does not work on Windows. I recommend passing include_dashboard=False when calling ray.init()