Wgpu: Functionality of HUB

[Michael-Lfx]

After a few hours reading context of HUB, I think it just convert all the void * to the right &'a (mut) T. Is it right?

Another problem is that there is no code completion with open with CLion.

[kvark]

HUB doesn't convert void pointers to mutable references, that sounds incredibly dangerous :)

HUB is just a globally shared structure that registries per resource/object type that need to be accessed by opaque user-facing IDs. Each registry is wrapped in a Mutex, so we lock them individually on access. See https://github.com/gfx-rs/wgpu/blob/master/wgpu-native/src/registry/mod.rs.

[grovesNL]

For the local registry we do use void pointers sometimes internally to create opaque types without the overhead of a hashmap or similar.

Only the remote registry uses a mutex at the moment, although I guess we might still need to add synchronization to some local registries eventually, depending on how threading works (either synchronizing fields individually, or an entire struct).

[kvark]

Oh snap, thanks for correction! Most things should be accessible from multiple threads, so we need those locks to be real. Also, we need to make sure that we lock in the same order everywhere, so that no deadlocks happen.

[Michael-Lfx]

Sorry for the chaos. I should stress I am referencing local mod not the remote one.
In further detail when build with the local feature the HUB will occupy zero bytes in theory because they just own PhantomData? @kvark

[Michael-Lfx]

although I guess we might still need to add synchronization to some local registries eventually,

Sorry I am confused. I think that for the local feature HUB is a central adapter which switches void * to &(mut) T.
So why is adding synchronization for local registries?

depending on how threading works (either synchronizing fields individually, or an entire struct).

The remote currently is using the former strategy, right? Comparing to the latter, synchronizing fields individually will get a better performance? How to ensure data safe of the entire struct? @grovesNL

Thank you for taking the time to answer my questions.

[grovesNL]

I'll try to clarify what I meant, there are a few separate synchronization points we're interested in here:

• Synchronization of registries for addition/deletion of tracked objects, for example, a new texture is created and we want to track it.
  
  
  
  • For local, we return an Id, which happens to be a void pointer to our newly created struct. When we want to use it later we will convert it back to a reference.
  
  
  • For remote, we also return an Id, but now it's an unsigned integer that we insert into a shared hashmap with the newly created struct. Later we can use to look up the struct in this shared hashmap.
  
  
  • The synchronization here is mostly relevant for remote usage because we use the shared hashmap, and we can't write to it at the same time on multiple threads.
  
  
• Synchronization of shared tracked objects, for example, submitting command buffers recorded on multiple threads to a shared queue (in this case, the queue is tracked)
  
  
  
  • For both local and remote we access and mutate a shared queue. We need at least one mutex in this case. We also might not want to rely on the synchronization at the registry level (for remote), so the registry itself might contain Mutex<Queue>, for example.
  
  
  • If a struct is mostly read-only but contains a shared field (for example, maybe error tracking that can be written by multiple threads), then we might want to just have a mutex around a single error field, or similar.
  
  

(when I say "shared" above I mean the object is visible to multiple threads)

So the synchronization of individual structs/fields is dependent on which tracked objects will be shared on multiple threads. In contrast the synchronization at the registry level is mostly related to our shared hashmap, but could be used to indirectly enforce synchronization for the structs/fields too (if we don't need the granularity).

@kvark can correct me if my understanding is wrong or I've missed anything :)

[kvark]

I actually totally forgot that we have bare pointers (as you can see by my earlier comment here ^) and thought we are working with mutexed registries. We should totally disallow get_mut on registries which we can't enforce to be accessed from a single thread only.

[grovesNL]

@Michael-Lfx Is this issue resolved? We'll continue to update hub/registries as we improve multi-threading support. Please let us know if you have any more questions 😄

[Michael-Lfx]

Yes. Thank you for answering my dumb questions. Love you guys.