Uvloop: libuv filesystem async calls

[pohmelie]

AFAIK, libuv also support async filesystem calls. Can this be implemented in uvloop (or based on uvloop) with nice await with and some pathlib awaitable coroutines? If so, «not good» executors for filesystem calls can be dropped :clap:

[1st1]

Yes, once I'm done with adding/testing the basic APIs, we'll experiment with files ;)

[pohmelie]

Great!

[GMLudo]

For my information, behind the scenes, it uses what on Linux? Threads?

[1st1]

It uses threads on all platforms -- that's the only reliable way.

[pohmelie]

@1st1 hm, if so, is there any profit of use libuv for file io?

[1st1]

I've no idea. I think it will be a bit faster, since IO threadpool in libuv is implemented in C (compared to pure Python executors in asyncio).

[pohmelie]

@1st1 sounds right. And I answer my question: this will be the only solution (with nice syntax of course), since AFAIK there is no async filesystem library in python except aiofiles (but this have no path operations).
Thanks for great project! :+1:

[saghul]

FWIW, the current builtin threadpool just has 4 threads.

[1st1]

@saghul Is that usually enough (I assume getaddrinfo uses the same threadpool)? Would be great to have an API call to change the size, instead of the UV_THREADPOOL_SIZE env var...

[saghul]

So far it has been... unless you use it for long running blocking operations like talking to serial devices. We have plans to improve it, but no real ETA.

[GMLudo]

@saghul: a filesystem stored on a slow SAN because it's overloaded, could also be considered as a long running blocking operation like serial, or not?

[saghul]

Yep.

[1st1]

@GMLudo That's not a problem atm, since uvloop doesn't have any methods to work with files.

[1st1]

I'll just close this. First async file IO must be implemented in asyncio directly.