Stl: shared_ptr idle deletion

[ohhmm]

std::shared_ptr use is scope-agnostic which allow us to dispatch its object deletion on an idle priority thread.

[StephanTLavavej]

Is there a question here?

[ohhmm]

Wouldn't it be great to have it in a way in a standard?

[AlexGuteniev]

I don't think so, such a facility is not general purpose enough.

[ohhmm]

Please, @AlexGuteniev,

Could you describe a case where it may be not an option?

Thanks,
Serg

[miscco]

I think "idle priority thread" is already something that cannot be guaranteed.

What the standard needs are hard guarantees that are always fulfilled on essentially all platforms. Just because there might be some platform that could optimize behavior under some conditions, that is nothing the standard could endorse.

It is the same story like "unordered_map" is so terrible. It is not, it is just that the guarantees it provides are mostly unneeded. However, there are still valid use cases so just dropping them is a no go.

If you really have sufficient knowledge about your specific system and you really need to squeeze out the last cycles, you must write your own specialized implementation.

[miscco]

Also you could easily imagine some terrible code being written like:

std::weak_ptr<Meow> weakPtr;
{
    std::shared_ptr<Meow> sharedPtr{};
    weakPtr = sharedPtr;
}

if (auto locked = weakPtr.lock()) {
    // you do not want to end up here
}

I am not sure how you would guarantee that sharedPtr is destroyed in time

[ohhmm]

I think "idle priority thread" is already something that cannot be guaranteed.

What the standard needs are hard guarantees that are always fulfilled on essentially all platforms. Just because there might be _some_ platform that could optimize behavior under _some_ conditions, that is nothing the standard could endorse.

It is the same story like "unordered_map" is so terrible. It is not, it is just that the guarantees it provides are mostly unneeded. However, there are still valid use cases so just dropping them is a no go.

If you really have sufficient knowledge about _your_ specific system and you _really_ need to squeeze out the last cycles, you must write your own specialized implementation.

Just like SMP. But we have std::thread and std::async in standard. I think that if OS has multi-thread then it has idle priority. Please, correct me if you know cases of multi-threaded OS which has no idle priority scheduling.

[ohhmm]

Also you could easily imagine some terrible code being written like:

std::weak_ptr<Meow> weakPtr;
{
    std::shared_ptr<Meow> sharedPtr{};
    weakPtr = sharedPtr;
}

if (auto locked = weakPtr.lock()) {
    // you do not want to end up here
}

I am not sure how you would guarantee that sharedPtr is destroyed in time

Once object dispatched for deletion, it considered as deleted in current semantics. This code would work correctly.

[MikeGitb]

@ohhmm:

The way shared_ptr is currently specified, it gives me hard guarantees, when an object is destroyed, which can be important, because the destructor may have side-effects (like unlockign a mutex or releasing other resources) and at the very least it has an effect on the available memory on my system. If you shedule deletion on an idle thread you a) have no control over when the destructor is run and b) you might actually have races if the destructor modifes some state that is also accessed from another thread (if nothing else, it is quite likely that the destructor accesses a global allocator).

That isn't to say that there aren't many valid usecases for this style of garbage collection. It just doesn't fit with the specification of std::shared_ptr, so this is nothing the STL maintainers can implement, even if they thought it was a good idea.

If you think the standard library should provide a type with those semantics (it would have to be a separate type, because it would be a breaking change for shared_ptr), you need to write a paper and convince the committee of it (for the record I'd say the chances are rather slim, but I've never been to a meeting, so what do I know).

[ohhmm]

Thanks @MikeGitb,

It makes sense.
I wouldn't tell about hard guaranties though because shared_ptr is multithreaded and another thread may be suspended by OS.
But overall your opinion seem inline with potential committee opinion.
I was asking it here to see if it gets any support by Microsoft.
Usually distinct person would not write a proposal to committee.
It guided by big companies.
I am trying to seed this idea.

Thanks,
Serg

[KubaO]

std::shared_ptr use is scope-agnostic

The objects it may delete don't have to be and generally aren't.

dispatch its object deletion on an idle priority thread

This requires that all destructors for classes used with std::unique_ptr are thread-safe. C++ doesn't require this of course. Generally speaking, this is unlikely to be the case. Several projects that I work on would instantly break if this idle deletion was done. We have asserts in the destructors that will fail if the object has been moved between threads - for objects that don't support thread-safe destruction.

I think the only way to make it work would be to add a special trait that enables a given type to be deleted this way. But whether it's worth it - I wouldn't know. I can't think of a situation where this would uniformly help - and I imagine on any busy system this would only make things worse, since now we'd have destructors jumping between cores and missing caches. This might even spill into the allocator and increase time-under-lock (if locks are used - it wouldn't affect lock-free allocators much).

[ohhmm]

I seen profiling results where shared_ptr destruction takes about ~30%
Rust has asynchronous destructors feature already BTW.
It is totally safe and it totally makes sense in all cases.

[KubaO]

It is totally safe and it totally makes sense in all cases.

It absolutely isn’t totally safe if you didn’t design the destructors in the entire inheritance hierarchy for a given class to be thread-safe. It’s trivial to break it so the assertion that it’s “totally safe” holds no water. Yes, shared ptr is thread safe by itself. That’s a requirement but not sufficient. The objects being destroyed must support being destroyed in another thread, and that is absolutely not generally the case nor guaranteed nor can be depended upon. At best it can be an opt-in for some class hierarchies.

Even if you run the destructor immediately but defer the memory release till later, you may be killing performance of memory allocators that were designed to leverage thread locality.

So, generally it’s a no-go, but if you wanted to implement it for some classes in your project, it’s like a page of code and ten pages of tests - no big deal.

[ohhmm]

I am still 100% sure it is totally safe.

[AdamBucior]

I am still 100% sure it is totally safe.

No it's not. There are objects that need to be created and destroyed on the same thread (for example a window in Win32 API).

[sylveon]

Another example is anything XAML-related (in UWP or the upcoming WinUI 3) must only be accessed from the GUI thread. Destructors performing cleanup from a background thread would irrecoverably crash the app.

Unlocking a SRWLOCK must be done on the same thread that locked it - so if the destructor unlocks one, you're done too (binary semaphores tend to be the only thread synchronization primitive which support locking on a certain thread and then unlocking on another).

If the destructor accesses global data, this change can introduce new data races silently.

While your idea is a pretty good idea in theory, a generic-purpose library like the STL can't support it because a lot of code it supports relies on destructors being ran on the same thread. IIRC Rust's async destructors don't necessarily specify destruction on a background thread - it is very much possible to be asynchronous on a single thread.