Tokio: Task local storage?

[vorot93]

This may be very useful for logging and tracing.

[Mart-Bogdan]

Indeed this is very useful for logging!

So-called pattern NDC -- nested diagnostic context, where you could hold:

• trace-id
• http request id
• user id
• current operaion name
• etc

And include it into logging output.

[mexus]

Won't it vendor-lock code that would utilize this mechanism to tokio? Because other runtimes might either not implement such a storage at all, or implement it in a different manner.

.. not to mention it will be yet another library-dependant side effect :)

Edit:
Just to clarify: I'm not against task-local storage, and I'm actually also fine with vendor-locking to tokio, but I'd rather use some more generic way to solve the aforementioned problems (trace id, user id, etc.).

[Mart-Bogdan]

If I recall correctly, Old futures had such mechanism, was it stripped from new futures spec?

[Mart-Bogdan]

Yes, it was in futures-0.2.0 but are stripped in the latest version

Context::with_locals and LocalKey

[davidbarsky]

For the use-case (nested diagnostic context) you're describing, would tracing's spans be an acceptable workaround? A request ID can be added to an instrumented future's span, and futures can be instrumented using the #[instrument] attribute macro or the instrument combinator.

[Mart-Bogdan]

@davidbarsky does span.in_scope (|| works for async functions?

I guess user is required to pass around this spans in all functions/methods :(

[Mart-Bogdan]

Also async-local cold be used to hold database transaction, with automatic commit/rollback after future completion.

But such usage could be considered anti-pattern by someone, aka dreaded ambient-context.

But I consider it quite usefull, dotnetcore is using it internally to hold connection from pool and other stuff, but encourages developers to not to use it directly, only in library code.

BTW. async-std has such feature https://docs.rs/async-std/1.1.0/async_std/task/struct.LocalKey.html

[davidbarsky]

@Mart-Bogdan You can instrument an async function in one of two ways:

#[instrument]
pub async fn my_function() -> Result<(), ()> {
    // ...
}

use tracing_futures::Instrument;

let my_future = async {
    // ...
};

my_future
    .instrument(tracing::info_span!("my_future"))
    .await

The #[instrument] attribute macro is just sugar for the the instrument combinator.

[Mart-Bogdan]

Oh, thanks for the example, just hadn't dug into tracing yet.

[davidbarsky]

The other thing that I'll note is that the instrument combinator—in whatever form you use it—is attached to the _future_, not the task. Not every async function will result in a new task.

[hawkw]

I guess user is required to pass around this spans in all functions/methods :(

@Mart-Bogdan tracing spans are ambient and don't have to be passed around — this is one of the slog pain points we wanted to avoid :) Normally, you would set the current span context using a drop guard, but, since async functions don't play nice with drop guards, you can use the instrument combinator instead.

The other thing that I'll note is that the instrument combinator—in whatever form you use it—is attached to the _future_, not the task. Not every async function will result in a new task.

Yeah, I'll add to this that someone recently tried to implement a tracing subscriber that tracked the current span using async-std's task locals, but they found that task locals offered insufficient granularity to represent traces where a task might involve multiple spans. It's possible other tracing systems can use task-locals, if you don't have nesting of spans within one task.

[TrionProg]

Hello, I really need Local Storage.

Do you remember Erlang/OTP? Each process has ID, Name, Receiver(MailBox), knows children, it spawned. I've already made my platform and spent 2 years.. but with old futures. Now I can not port in on 3th version.

As I see, request to LocalStorage/Variable is not fast function, that is why I must drag pointer on it to any future. In most cases I can do it, but some futures may have no generics fields, where I can store this pointer, I will lose it and later I will need it again, how can I take it without global storage for task?. But In my case all is more difficult and I do not know how to avoid this problem, especially if code will be generated or universal. And I want do not make code dirty by this pointer, I just have ability to hide it, but only in my case, and it do not fixes problem with loosing the pointer by future.

I hope, you will fix it.
For exampe, 'Clever' Task you can find here https://docs.rs/async-std/1.3.0/async_std/task/struct.Task.html But I still not used this library and I do not know about compability with tokio.

[carllerche]

This has been released: https://docs.rs/tokio/0.2.22/tokio/macro.task_local.html