Android-components: New component: Nearby

[jonalmeida]

We want to be able to find devices in the area that we may not already be paired to so we can share sessions.

This component would mostly be for discovering/connecting to other devices. The data itself can be figured out later (separately?).

[jonalmeida]

EDIT: Renamed 'Share Manager' to 'Nearby Manager' to avoid immediate confusion with android sharing.

There are two major parts for local sharing controlled by the main entry point, Nearby Manager:

1. Discoverability
2. Transfer Protocol

The 'Nearby Manager' co-ordinates the Discovery Manager and the Tunnel Manager and allows the app to be notified of connections established/failed and turning services on/off.

graph TD;
    A[Nearby Manager] --> B[Discovery Manager]
    A --> C[Tunnel Manager]
    B --> D[Wifi]
    B --> E[Bluetooth]
    C --> F[Wifi]
    C --> G[Bluetooth]
    F --> H[Server]
    F --> I[Client]
    G --> J[Server]
    G --> K[Client]

Discoverability

Discovery Manager

• Controls which bearers are turned on/off.
• Manages/Listens to those bearers to notify listeners that we have
• This is how the device broadcasts the services available for transport.
• WiFi uses DNS-SD (local network service discovery) - it broadcasts the services it has which other devices listen to.
• Bluetooth has service scan filtering by UUID - we use a hardcoded UUID to scan for across devices.
• These service discovery bearers need to be abstracted out so that if we want to add more we can do that in the future.
• We would hard code a sensible bearer priority if multiple are available. (e.g. Wi-Fi > Bluetooth > USB)

Interface

interface DiscoveryService {
  fun scanningStarted()
  fun scanningStopped()
  fun deviceFound(Service device)
  fun deviceLost(Service device)
  val bearerType: ServiecType
  val deviceName: String /* Update discovable name of device */
}

sealed class Service(val deviceType: DeviceType) {
  data WifiDevice(val uuid: String, val host: String, val port: Int)
  data BluetoothDevice(val uuid: String, val macAddress: String)
}

enum class ServiceType {
  Wifi,
  Bluetooth,
  Unknown
}

enum class DeviceType {
  Mobile,
  Desktop,
  Unknown
}

class DiscoveryManager(discoveryBearers: List<DiscoveryBearer>) {
  fun init(enableWifi: Boolean, enableBluetooth: Boolean)
  fun enableService(/* TBD */)
  fun disableService(/* TBD */)
  fun deviceFound(device: Bearer, type: BearerType)
  fun deviceLost(device: Bearer, type: BearerType)
}

Transfer Protocol

Tunnel Manager

• V1:

  • For each bearer that broadcasts itself, we need a transport that would allow us to transfer data across it - 1-1 mapping.
  • WiFi uses mDNS to broadcast; WebSocket server for transport layer.
  • BluetoothLE to broadcast; BluetoothServer for transport layer.
  • When in "discovery" mode, we should be starting a the same service's tunnel server for accepting connections.

• Future:

  • Discovery and transport don’t need to be tied together. e.g. Discovery on bluetooth, transport over WiFi Direct.
  • More discovery possibilities; Google's Nearby API, USB, quantum tunneling.

interface TunnelServer {
  fun start() 
  fun stop()
  fun tunnelOpened(tunnel: Tunnel)
  fun tunnelClosed(tunnel: Tunnel)
  fun broadcast(data: ByteArray) = Unit // not required
  fun broadcast(data: String) = Unit // not required
}

interface Tunnel {
  fun close()
  fun send(data: ByteArray)
  fun send(data: String)
  fun isOpen()
}

abstract class TunnelClient implements Tunnel {
  abstract fun onConnect(tunnel: Tunnel)
  abstract fun onClose(tunnel: Tunnel)
  abstract fun onMessage(tunnel: Tunnel, data: ByteArray)
  abstract fun onMessage(tunnel: Tunnel, data: String)
}

class TunnelManager(val servers: List<TunnelServer>) {
  fun init() = servers.forEach{ it.start() }
}

Nearby Manager

Connects the DiscoveryManager and TunnelManager together.

class NearbyManager(
  val discoveryManager: DiscoveryManager, 
  val tunnelManager: TunnelManager
) {
  val priorityList: List<BearerType> = listOf(Wifi, Bluetooth)
  val deviceName
    get() = field.value
    set(value) { /* notify discovery service */ }

  /* various notifiers for the app; tbd */

  discoveryManager.onServiceSelected ( service -> service.connect() )
  tunnelManager.onConnected { tunnel -> notifyObservers { connected(Tunnel) } }
}

User Flows

1. User A has Wi-Fi; app has enabled Wi-Fi service:
   
   
   
   1. Wifi tunnel server started.
   
   
   2. User A discovers User B.
   
   
   3. User B selected and WebSocket client connection attempted and established (app notified to send data)
   
   
2. User A has Bluetooth; app has enabled Bluetooth service:
   
   
   
   1. Bluetooth tunnel server started.
   
   
   2. User A discovers User B.
   
   
   3. User B selected and Bluetooth client connection attempted and established (app notified to send data)
   
   
3. User A has Wifi and Bluetooth enabled, app has enabled both services:
   
   
   
   1. User finds and selects User B on Wi-Fi and Bluetooth.
   
   
   2. Nearby Manager considers the priority list to choose which service to use.
   
   
   3. Connects to [insert higher priority device].
   
   
4. User fails to connect to a service:
   
   
   
   1. User A finds and selects User B.
   
   
   2. A client connection attempted and failed (app notified of failure).
   
   
   3. Note: Do we try again? If there are multiple services available, do we try the second one?
   
   

[jonalmeida]

Other things to consider:

• https://wiki.mozilla.org/FlyWeb - Similar concept and (wifi) discovery, but baked into Gecko (and now no longer active?)
• None of these names/methods are final. I'm sure there are better names instead of 'Tunnel Manager' 😉
• Not sure what's the right way to build a secure websocket with certificate pinning (i.e. providing a prod version, easy local development)

[pocmo]

Thank you, @jonalmeida.That's a great detailed issue. :)

My assumption from the mocks is that a "neary" feature will be a killer feature once it is supported on all devices (including desktop and iOS). Did you research this a bit? Does iOS have some capabilities built-in (like Google's nearby API?)? Getting the desktop world onboard may be harder - did you reach out to any teams yet?

[jonalmeida]

Does iOS have some capabilities built-in (like Google's nearby API?)?

iOS supports mDNS as well so we'd be able to use that for Wi-Fi i.e. this could be an iOS component that uses the same mDNS service type to discover over Wi-Fi, and WebSockets to create a server/client connection.

For Bluetooth, I assume there is a similar BLE API, but I haven't looked into it yet.

Getting the desktop world onboard may be harder - did you reach out to any teams yet?

I haven't reached out to the FlyWeb folks yet, but I'll make it a point to do so and see what their thoughts are.

That's also a point I forgot to make: this implementation would not be platform-specific, we're using open standards so everything here is just implementation details that we'd have to share across other platforms. Scoping this to work for all platforms with just Wi-Fi is probably a more achievable task.

[jonalmeida]

I spoke with @justindarc about the FlyWeb implementation and got some useful insights out of it:

• mDNS implementation exists in Gecko already: uses the platform's mDNS responder otherwise falls back to a JS implementation.
• It's pretty powerful already for browser-to-browser connections.
• Included a web server for handling connections.
• This is a very similar design to what I proposed above.
• Code: https://searchfox.org/mozilla-central/source/netwerk/dns/mdns/nsIDNSServiceDiscovery.idl

Drawbacks:

• The implementation is HTTP only (this may have changed recently?)
  
  
  
  • Using self-signed certs was the next step in securing this with a Bluetooth pair-esque securing process. More solutions exist here.
  
  
  • There's a more detailed security overview as well.
  
  
• Baked in gecko means that it wouldn't support SystemEngine applications.
• mDNS only - doesn't have support for bluetooth or extensions for other network bearers.

Misc Notes:

• We can have an implementation of the Gecko version in addition to our native solution if/when it can get to a secure place that we're willing to ship it.

One of the things that helped us get past this [securing a connection] problem though was a notion of trust after the first connect. So, if I connected to device XYZ once, it could be trusted to connect subsequent times thereafter but, that first connection to device XYZ could be compromised.

This is a useful implementation idea if we decide to have a concept of trusted devices. It also brings in more questions about how do we associate trusted devices with synced devices if there are overlaps that requires discussion with FxA.

[justindarc]

One point of clarification to the above -- the mDNS interface is still in the mozilla-central tree, but the "web server" part of FlyWeb has been removed AFAIK. So, unfortunately, that means that the browser-to-browser demo video linked above will no longer run in Firefox Nightly. However, like I mentioned, all the mDNS/DNS-SD (service discovery) APIs should still be there as I believe they were also being used for other things besides FlyWeb (such as the Presentation API and some of the Dweb work).

Also, I believe the formal name for the trust model described above is Trust On First Use (TOFU). This security model is widely used today for lots of things including SSH (i.e. when you get prompted to trust the key of the server on the first connect).

[jonalmeida]

How do we avoid spoofing attacks is another big security question that I'm unsure off that probably needs a security review to get solutions on.

Focus on getting the functionality working first will probably be better.

[jonalmeida]

Random thoughts: this would make sense to have this made into a rust component similar to how application services are doing it - bindings for native applications, single logic source.

With the success of concept-fetch and the GeckoViewFetchClient we could do something similar here as well. It would mean that browsers with gv or supported gv multicast could use that implementation or fallback to the Android native/Google Nearby™️ implementation.

Downsides would be longer development and integration time. I'm not sure how much we're willing to invest on this.

[pocmo]

@jonalmeida I'll remove the fenix label here. This seems to be not in scope for MVP. :)

[pocmo]

Closing for now. We just removed one nearby component. Let's open a new issue once we need it again. :)