Mixedrealitytoolkit-unity: Which one is the most reliable ? Unet sharing or HTK Sharing

[saidbet]

Hi,

I've been working on a project for 4 months, and we've used the HTK sharing to share Holograms. We have a lot of Holograms to share, and we also made made a custom event trigger system that allows us to trigger events and send data from one device to another.

Everything works fine with 2 or 3 devices, but once we have more than 3 devices, the SharingService is unreliable and we have a lot of crashes.

I guess the problem is due to the high traffic between so much devices, so before starting to change everything, I would like to know if any of you have worked with the Unet sharing ?

If so, how reliable is it ? did you encounter any problems with it ? Is it stable with a lot of devices ?
Every bit of information would be usefull to me, so please share your experience.

Thank you.

[StephenHodgson]

That's a very good question. Personally I've seen up to 5 devices work really well using the Sharing Service. I'm currently working on a project where we're going to have up to 25 devices using the same network at once, and we will be testing that out pretty soon.

Some things to take into consideration with the Sharing Service: Network infrastructure and bandwidth is critical; Make sure you're not sending more info that you need across the network at once; Use the sync system (See SharingSpawnTest example) over sending messages at a low level (See CustomMessages class in the SharingTest example).

As far as UNet, I've no real experience with that service. We've stayed away from it because of the potential for Unity to make it cost money for match making, and the limit on concurrent users.

[saidbet]

In the current state, we have 3 hololens and 2 computers.
The 2 computers are kind of spectator views, so we use MRVC because we don't need more than 720p and the Spectator View is not ideal when you want to move around with it.

In addition to that, we have a "SyncMessage" GameObject for each device, and each device has a reference to other devices' SyncMessage. That GameObject is used as an event trigger, and as a simple way to send data along with the event. This probably causes a lot of traffic.

We use this system for visual effects like highlights, and click effects, and also for a bunch of other stuff, because sharing the transforms of nested GameObjects is a bit complicated: for example, when you have let's say a menu, and this menu has buttons as child GameObjects, and you want to share the transform of the menu as well as the transform of the buttons, well you can't because when a new device connects to the session, it will try to instantiate the child GameObject before the Menu GameObject (which is the parent).
To sum up, it will instantiate the GameObjects in reverse order (last spawned object is instantiated first) or maybe in random order, I'm not really sure, but everytime we've tried, it tried to instantiate the child before the parent and raised an exception because the parent didn't exist yet.

For the spectator side, the Hololens sends the video stream directly to the computer without going through the SharingService, so this shouldn't be the problem.
The server is running in the cloud, and we have at least 50 mbps up and down on each side, I think it should be enough.

It would help to have the source code of the SharingService to see where the problem comes from. Do you know if it's available somewhere, or do you have any contacts that could provide it to us?

[StephenHodgson]

https://github.com/Microsoft/HoloToolkit

[saidbet]

Wow thanks ! Didn't know it was there 👍

[StephenHodgson]

Sounds like you guys are a little over engineered. I'd suggest not making the menus and effects networked, but instead only drive those things locally. Only network the objects you _know_ other people will need to see changed, or interact with together. impo, a users UX should only ever be locally driven, with information populated from your network.

[saidbet]

The main purpose of the application is to be shown at a convention, so our client asked us to show as much as possible, even the menus so that spectators can see what's really going on in front of the guy wearing the Hololens.

[sebrk]

I cannot get anything with UDP to work. Do you guys have Auto discovery working? I've tried both my own implementation (which works fine on Windows 10 and other devices) and HoloToolkit examples. It seems direct IP is the only thing that works?

Because I'm sending very very little data (I just want to manage scenes loading) I don't really need a dedicated server as the SharingService. It seems this is also the only way for me to communicate with the Hololens (SharingService + direct IP). Have you guys ever gotten a simple UDP packet through to the Hololens? I can see the messages on the network btw. Also no firewalls anywhere (except possibly the Hololens, which I am unaware of if it even has one).

[saidbet]

We didn't manage to get Auto Discovery working. We have one device that sets the Sharing Server's IP, and then, broadcasts a message with that IP to the other Hololens.
Since you only need some casual data transfers, maybe you could use that:
https://docs.unity3d.com/ScriptReference/Networking.NetworkDiscovery.html

[sebrk]

Thanks for the reply @saidbet.

Thing is I've tried just about everything. Hololens doesn't respond to anything even though it works fine between other devices (UWP). Can you share your broadcasting code so I can test it? I'm really desperate. Need to figure out if there is something wrong with my equipment.

[Ziugy]

I've been using Unet for the majority of projects. Had 4 devices reliably sharing. Adding a 5th device had a few slow downs, but didn't have time to iron out any networking optimizations before needing to move onto the next project. We networked UI and many hologram positions.

Another note: Spectator View had a PR that went into master rather recently that started leveraging Unet. Therefore you may want to do investigations on this before switching.

[sebrk]

The HoloToolkit solution was way overhead for me. I decided to implement it myself using UNET. But then again, I basically send control messages every now and then.

Could never get the HoloToolkit version to work well either. I now have auto discovery working with a workaround where the hololens acts as both server and client before passing control to the actual server. Weird but it works.

[HattMarris1]

@StephenHodgson any chance you could share some more details on the project you mentioned having up to 25 devices? I might soon be working on a project having up to 15 devices on the sharing service, and I'd be interested to learn how difficult it was.

[sebrk]

There seems to be a lot of weird UDP issues on Hololens. Sometimes it works. Sometimes not. Are there any official word on why UDP behaves this way?

[StephenHodgson]

Going to close this. It's probably been archived by google by now.