Webrtc-pc: "a=msid" line should contain sender/receiver IDs, not track IDs

[henbos]

Traditionally "a=msid" lines indicate tracks and streams by track IDs. With RTP Media API, including addTransceiver and replaceTrack, it makes more sense to think of these as sender/receiver IDs.

On the receiving side, an RTCRtpReceiver will be created whose track has the announced ID, and it's receiving media from an RTCRtpSender on the other side. But the sender can replaceTrack, there is no guarantee that MediaStreamTrack IDs match on local and remote sides.

Traditional case:
pc.addTrack(track); // a=msid with track.id
Here, the local and remote side will get MediaStreamTracks whose id match.

But what about this?

senderA = pc.addTrack(track);  // a=msid with track.id
O/A
senderA.replaceTrack(null or something else);
senderB = pc.addTrack(track);  // a=msid with...?

Or this?
pc.addTransceiver('video'); // track is null by default, a=msid with...?

Proposal: When an RTCRtpSender is created we should assign it a unique ID and use that for the "a=msid" line. The only MediaStreamTrack.id we can guarantee something about is the one used by an RTCRtpReceiver - the receiver and its track can share ID.

[henbos]

If this seem to make sense to you I can create a PR.

[fippo]

if you give up the notion of putting the track ids into the sdp even for the most simple cases that do not use replaceTrack at all then you can just rely on the transceiver.mid so the track.id would be something like send- + mid and recv- + mid.

See @jan-ivar's blog post:

Correlate by transceiver.mid instead of stream/track.id.
If you’re sending track ids out-of-band today hoping to correlate local 
and remote tracks, WebRTC 1.0 will break you.

[stefhak]

I think we have a few options in this domain (track/sender/receiver ids):

1. leave things as they are (with a=msid used to carry the track id over to the track created at the remote side)
2. redefine so that the id of the remotely created track is generated there (i.e. no correlation with the sender at all). Not clear what we do about a=msid.
3. have the sender generate an id (readonly attribute on sender I presume), which is signaled with a=msid. I guess this is what is proposed by @henbos
   3.1 Is the same id is reflected on remote end receiver (readonly attribute)?
   3.2 Is the same id is reflected on remote end track, or would that have a locally generated id?

Remember that a=msid also signals _MediaStream_ id, and I've not heard us discussing getting rid of that one. And, as pointed out, mid can also be used to correlate.

I'm somewhat afraid of what ramifications a change would have (thinking about JSEP, msid I-D).

[fippo]

jsep says this about subsequent offers:

For RtpTransceivers that are not stopped, the "a=msid" line(s) MUST stay the same if 
they are present in the current description, regardless of changes to the
transceiver's direction or track

Does the id (which is either the initial track id or a randomly generated one) need to be stored in an internal slot?

[henbos]

@stefhak:

3.1 Is the same id is reflected on remote end receiver (readonly attribute)?
3.2 Is the same id is reflected on remote end track, or would that have a locally generated id?

The receiver always has a track, which never changes, so whether we store this ID (same as other endpoint's sender ID, i.e. the "a=msid" line) as the receiver's track's id or have an receiver [IdInternalSlot] does not matter unless we think there is a risk future changes to the spec might want to replace receiver's track as well. (An internal slot would be more future-proof.)

Or, rather, in accordance with @fippo's jsep quote...

For RtpTransceivers that are not stopped, the "a=msid" line(s) MUST stay the same if
they are present in the current description, regardless of changes to the
transceiver's direction or track

...the [IdInternalSlot] belongs to the transceiver, since in all cases we have both a sender and receiver.

---

In response to the other options:

1. leave things as they are (with a=msid used to carry the track id over to the track created at the remote side)

This does not work. In the examples I gave above this would create multiple "a=msid" lines with the same ID or undefined behavior for addTransceiver() with null track.

For backwards-compatibility reason, any option we do choose can default to use the track ID as the transceiver ID iff it has not already been used, and generate a new one otherwise. This would not break apps that make assumptions about track IDs that never do replaceTrack or addTransceiver.

1. redefine so that the id of the remotely created track is generated there (i.e. no correlation with the sender at all). Not clear what we do about a=msid.

I think it is worth discussing whether the receiver's track ID should match the "a=msid" line or not, but it is clear according to the above jsep quote that it has to correlate to a transceiver.

[henbos]

Remember that a=msid also signals _MediaStream_ id, and I've not heard us discussing getting rid of that one. And, as pointed out, mid can also be used to correlate.

(I need to read up on this.)

[fippo]

senderA = pc.addTrack(track);  // a=msid with track.id
O/A
senderA.replaceTrack(null or something else);
senderB = pc.addTrack(track);  // a=msid with...?

This will end up on a different m-line / transceiver.

pc.addTransceiver('video'); // track is null by default, a=msid with...?

http://rtcweb-wg.github.io/jsep/#rfc.section.5.2.1 -- If no MediaStreamTrack is attached, a valid ID MUST be generated, in the same way that the implementation generates IDs for local tracks.

[henbos]

OK cool. Then the only problem is this:

For each MediaStream that was associated with the transceiver when it was created via addTrack or addTransceiver, an "a=msid" line, as specified in [I-D.ietf-mmusic-msid], Section 2. If a MediaStreamTrack is attached to the transceiver's RtpSender, the "a=msid" lines MUST use that track's ID. If no MediaStreamTrack is attached, a valid ID MUST be generated, in the same way that the implementation generates IDs for local tracks.

There is no clause for what if the track's ID is already in used by another transceiver (due to replaceTrack or addTransceiver). In that case we must generate an ID too.

[stefhak]

There is no clause for what if the track's ID is already in used by another transceiver (due to replaceTrack or addTransceiver). In that case we must generate an ID too.

Is something forbidding the use of the track's id? It would be a duplicate, sure, but I'm not sure that's a problem.

[fippo]

@stefhak good question. ontrack has the transceiver set so one can differentiate. But it gets confusing and will break current assumptions about ontrack

[jan-ivar]

See also https://github.com/w3c/webrtc-pc/issues/1702.

[alvestrand]

Waiting for action on https://github.com/rtcweb-wg/jsep/issues/842

[ibc]

Related question:

Why does the ontrack event come with a streams field (FrozenArray<MediaStream>)?

The streams attribute returns an array of MediaStream objects representing the MediaStreams that this event's track is a part of.

If the sender and receive tracks are decoupled then there is no chance for "duplicated" receiving tracks, so it's impossible that the same track belongs to more than one stream.

In the other side, I don't understand why we need the msid at all. Having a stream id within the transceiver is useful to group tracks into a stream. But we don't need the "more track id" at all...

[stefhak]

The streams attribute returns an array of MediaStream objects representing the MediaStreams that this event's track is a part of.

If the sender and receive tracks are decoupled then there is no chance for "duplicated" receiving tracks, so it's impossible that the same track belongs to more than one stream.

Actually, the sending side can create a number of (possibly) empty MediaStreams (new MediaStream()), and use these as argument for addTrack; at the receiving side the corresponding MediaStreams would be created, and the track would be a member of all of them.

[ibc]

Actually, the sending side can create a number of (possibly) empty MediaStreams (new MediaStream()), and use these as argument for addTrack; at the receiving side the corresponding MediaStreams would be created, and the track would be a member of all of them.

I meant assuming we no longer generate remote tracks with the id given in the remote SDP.

[stefhak]

Actually, the sending side can create a number of (possibly) empty MediaStreams (new MediaStream()), and use these as argument for addTrack; at the receiving side the corresponding MediaStreams would be created, and the track would be a member of all of them.

I meant assuming we no longer generate remote tracks with the id given in the remote SDP.

But still, addtrack(track, list-of-streams) would (eventually, after SDP O/A) lead to a remote track (even if its id has nothing to do with the id of the local track) belonging to all streams (which are re-created at the remote side) in list-of-streams on the remote side. What am I missing?

[henbos]

It is possible for the same track to belong to multiple streams by providing multiple streams to addTrack(), on the remote end a receiver is created with a track, and that track is added to all corresponding streams.

In the case of multiple senders the remote end gets multiple receivers and tracks, even if the senders all send the same track, so each sender describes a unique track-stream relationship for the receiver side.

I think this is as it should be and tangential to this discussion about ensuring transceivers have unique IDs.

[henbos]

It would be a duplicate, sure, but I'm not sure that's a problem.

If multiple transceivers have the same ID, how would you know which one of them are changing direction, parameters, etc, in a follow-up O/A cycle?

[ibc]

I think I missed the use case for addtrack(track, list-of-streams) (with more than a single stream).

This WebRTC dependency on the MediaStream artifact is and will be a pain for years. Just look at ORTC: no "MediaStream" in there. Just let the application signal the relationship between audio/video tracks in its own way. Browsers are not dumb phones that can just generate and consume SDP blobs.

[taylor-b]

If multiple transceivers have the same ID, how would you know which one of them are changing direction, parameters, etc, in a follow-up O/A cycle?

You'd typically identify transceivers by their order, or their MID, not the track ID value. So I don't think it's a problem personally.

[henbos]

I personally don't like order very much for readability but as long as you never remove a transceiver from any future offer it would work. I don't want to bikeshed.
OK - I'm ignorant about MID, I should read up on that.

[ibc]

I'm ignorant about MID, I should read up on that.

MID is just an unique identified per m= section:

a=mid:audio
a=mid:video
a=mid:foooobar1

[alvestrand]

1567 was merged and reverted at one time?

[taylor-b]

I think the resolution was basically "this is a band-aid solution for backwards compatibility, and you can work around it by just parsing SDP, so it's not worth doing".

[henbos]

Is the conclusion "we don't care about duplicate track IDs or changing them to be sender/receiver IDs because we already have MIDs which is unique per transceiver"?

[fippo]

track id will go away:
https://github.com/rtcweb-wg/jsep/issues/842#issuecomment-364840604

:clap: @juberti

[henbos]

What happened to this? I see the JSEP PR was never merged

[aboba]

@henbos Let's recap the situation at TPAC 2018 and figure out the next steps.

[henbos]

With the JSEP PR closed and transceiver already having an ID that does match on both endpoints ("mid") I'm closing this issue. If there are any follow-up discussions let's have them in https://github.com/w3c/webrtc-pc/issues/2005.