Twilio-video.js: Local participant's 'trackUnpublished' event not fired

[dferrazm]

It seems that the trackUnpublished event is not being fired for the local participant whenever unpublishing a local video track (didn't test for the audio track, but perhaps has the same issue). The trackPublished is fired fine on the local participant whenever a new local video track is published.

Remote participants' trackPublished and trackUnpublished also work fine.

Code to reproduce the issue:

localParticipant.on('trackPublished', ({ track }) => {
  console.log('track published:', track.id)
})

localParticipant.on('trackUnpublished', ({ track }) => {
  console.log('track unpublished:', track.id)
})

createLocalVideoTrack().then(track => localParticipant.publishTrack(track))
localParticipant.videoTracks.forEach(trackPublication => trackPublication.unpublish())
createLocalVideoTrack().then(track => localParticipant.publishTrack(track))

Expected behavior:

I'd expect the following outputs:

// track published: 111
// track unpublished: 111
// track published: 222

Actual behavior:

It seems that the unpublishedTrack event is not being fired.

// track published: 111
// track published: 222

Software versions:

• [X] Browser(s): Chrome 74, Firefox 67
• [X] Operating System: Ubuntu 18.04
• [X] twilio-video.js: 2.0.0-beta9
• [X] Third-party libraries (e.g., Angular, React, etc.): React

[dferrazm]

I've checked the source code, and I noticed that in the localparticipant.js, the trackUnpublished event is non-existent. I'm not sure if that's intended.

Perhaps what we're missing here is on line 477, right after the _removeTrackPublication call, having the following:

const removedPublication = this._removeTrackPublication(localTrackPublication);

setTimeout(() => {
  this.emit('trackUnpublished', removedPublication);
});

[dferrazm]

I can confirm that it worked. I could work on a PR if needed.

[manjeshbhargav]

Hi @dferrazm ,

Thanks for writing in with your issue. As of now "trackUnpublished" is fired only on RemoteParticipants. Right now, unpublishing a LocalTrack by a LocalParticipant is a fire-and-forget type of an action where there is no feedback from our backend about the result of this action. Is there any reason you want a "trackUnpublished" event on the LocalParticipant? If it is for updating the UI, you can assume that the LocalTrack is unpublished as soon as you call LocalParticipant.unpublishTrack().

I hope this answers your question.

Thanks,

Manjesh Malavalli
JSDK Team

[dferrazm]

Hi @manjeshbhargav.

Thanks for the reply.

The reason someone would expect the trackUnpublished event to be fired for the local participant is because of consistency, given that the trackPublished event is being fired. So it's reasonable to think that the counter-part event when unpublishing would also be fired. If the trackUnpublished event is something that makes no sense for the local participant, then I'd argue the same should be true also for the trackPublished event, and thus also not be present there. So either both exist or none of them exist for the local participant.

Yes, the problem it's related to the UI. I argue that both events (published/unpublished) should be fired for the local participant 'cause that allows low-coupling of UI components that handle different responsabilities in the view. I explain with my case (and possible other people's case as well, when moving from 1.x to 2.0):

My video chat app is a React app with highly decoupled-reusable components (which is good) that handle the different responsabilities in the video chat workflow, two of them being:

• ParticipantMedia: responsible to add/removing/rendering the (remote/local) participants tracks and which relies heavily on the events being fired when tracks are added/removed (which is now published/unpublished for 2.0)
• ScreenSharingToggler: responsible to toggling the screen sharing for the local participant, which means removing/unpublishing the camera feed track and adding/publishing the screen track and vice-versa.

Both components are decoupled, meaning that they don't know each other, meaning that whenever that the camera feed track is removed and the screen track is added, that would be picked up async by the local participant's ParticipantMedia component, through the trackAdded / trackRemoved events in 1.x. Allowing then the decoupling and ParticipantMedia and being reusable among local and remote participants, since, before, both participants would fired trackAdded / trackRemoved.

Now, with the 2.0 changes, we have two problems:

• it makes it difficult to achieve this decoupling and thus enforcing a bad architecture for clients when designing componentized UI, since the task of switching between video tracks would now be highly coupled with the task of rendering the tracks in the UI.
• lack of consistency and avoinding reusability, since, in 2.0, components responsible to rendering remote participants tracks can still rely on the publish / unpublish events to handle the remote tracks, but local participant's components can now only rely on the publish event to render the newly published tracks, but having now to rely on a global state or coupling solution to handle when the local participant's tracks are unpublished.

I'm pretty sure this a common scenario for clients of this lib implementing componentized web apps (with React or something else).

Lastly, I don't see the reason why the unpublishing is now a fire-and-forget kindof event for the local participant, since the same is not true for the publising event, and before, in 1.x, both trackAdded / trackRemoved (which is kindof what the new publishing events are replacing) would exist for both local and remote participants.

(sorry for the big explanation - just trying to make my point)

[thaihung019]

UPDATED: This not a bugs. This is my missunderstanding. Sorry about this. I think someone new will encounter this problem. Please message me when you face with this problem.

Hi @manjeshbhargav. My project is using web platform for desktop and mobile. I encountered a problem when receive remote video. Everything is perfect, except on Chorme browser it doesn't display remote video from iOS safari, but it works with remote video from safari of OXS from Macbook.

I also turned on the debug mode. The Video tracks and Audio tracks from remote is logged with value. I can heard the sound from IPhone but the video still not display.

However, sometimes, when I turn off the camera on IPhone, then turn it back on. The local video on IPhone is disappear, and the remote video of IPhone on Chorme appears with freeze.

Can you help me out of this problem, please?

By the way, thanks for the hard working of Twilio team. Your product is very good.

Software versions:
Browser(s): Chrome 74, Safari(iOS) 12.
twilio-video.js: 2.0.0-beta9
Third-party libraries (e.g., Angular, React, etc.): React

[dferrazm]

@thaihung019 I'm afraid this is a different issue than the one discussed here. I'd suggest opening a different issue for your problem so that things do not get mixed.

[thaihung019]

@thaihung019 I'm afraid this is a different issue than the one discussed here. I'd suggest opening a different issue for your problem so that things do not get mixed.

Hi Twilio team.

First of all, I really sorry about my problem. I just create duplicate local video track. That cause this problem. After taking time to review the code. I fixed it.

Thank a lot

[manjeshbhargav]

Hi @dferrazm ,

"trackAdded" and "trackPublished" are fired at different times. In 1.x "trackAdded" was fired as soon as you called LocalParticipant.addTrack() synchronously, where as "trackPublished" is called whenever we get feedback from the backend that the Track has been successfully published. We do not get any feedback from the backend when a Track is unpublished.

Since you are using React, you can still maintain decoupling between ParticipantMedia and ScreenShareToggler by storing the list of published Tracks in your app state (I'm assuming you're using Redux). Then, whenever you call unpublishTrack(), you can immediately update the list of PublishedTracks. Since this state is consumed by ParticipantMedia, it will then update its UI accordingly.

I hope my suggestion works for your use-case. Please let me know. Having said that, we will explore the possibility of adding a "trackUnpublished" event for the LocalParticipant.

Thanks,

Manjesh Malavalli
JSDK Team

[dferrazm]

Yes. I'm using React, but I'm not using Redux or any other sort of global state, as I'm trying to avoid it. The app is small and there was no need until then. I'm reluctant to add a global state now just because of this issue. I still think that the main problem here is consistency. If we have trackPublished being fired for local participant, it's reasonable to expect trackUnpublished as well, so a component responsible to handle these events can handle/work seamlessly for both track interactions.

I don't entirely agree that the argument "of not having the concept of track being unpublished for local participant in the backend" is a blocker to not have this event fired for the local participant. We have to think in the eyes of consumers of this lib, which sees it as a black-box and the signalling backend in this case is an implementation detail. In the standpoint of consumers of this lib, it's relevant to know that I'll get 'trackPublished' and 'trackUnpublished' whenever the track is actually published/unpublished. If this go through the signalling backend or not is not really relevant looking to it as a black box. As you already said that we can consider that, whenever the local participant unpublishes a track, we can consider the track already being unpublished right-away synchronously, it's safe to assume that we should get a 'trackUnpublished' event fired at that point. Again, going through the backend or not is an implementation detailed and not important for consumers of the lib. Having said that, perhaps it's not really even needed to introduce changes in the backend to have the track unpublished concept for local participants. I don't see the need for that.

wdyt?

[xdumaine]

I'm running into this as well. I'd like to have this event fired, also for react.

[ElliotChong]

Useful event for all the reasons @dferrazm called out- not having this event makes it unnecessarily difficult to notify parts of your application for this particular state change, especially when the rest of the SDK does a solid job of emitting events on state changes.

Hacky workaround:

const TRACK_UNPUBLISH_WORKAROUND = Symbol('trackUnpublishWorkaround')

function onLocalTrackPublished (publication) {
    if (publication.unpublish[TRACK_UNPUBLISH_WORKAROUND] !== true) {
        const { unpublish } = publication

        publication.unpublish = function instrumentedUnpublish (...rest) {
            const returnValue = unpublish.apply(this, rest)

            this.emit('trackUnpublished', publication)
            room.localParticipant.emit('trackUnpublished', publication)

            return returnValue
        }

        publication.unpublish[TRACK_UNPUBLISH_WORKAROUND] = true
    }
}

room.localParticipant.addListener('trackPublished', onLocalTrackPublished)

[HandsomeMedia]

Echoing the above -- it was frustrating to debug why unpublish does not fire an event while its publish counterpart does. +1 for firing an unpublish event as requested by @dferrazm

[dferrazm]

@manjeshbhargav I noticed the help-wanted label. Is this something I could help with? If you want, I can take over this issue and investigate how to better approach it.

[zoink]

just wanted to add a few notes - currently working on video calling app (react with fcs/hooks)
i wanted a basic enable/disable video + audio tracks functionality, but the track.enable() and track.disable() functions weren't working very consistently (prob due to a lot of shuffling participants around on my part)

so i just went with the safer/bulkier method of unpublish/publish tracks to mute and unmute
code looks like

// note: room is a ref
// mute
 room.current.localParticipant.videoTracks.forEach(
                                  (publication: any) => {
                                    room.current.localParticipant.unpublishTrack(
                                      publication.track
                                    );
// unpublishTrack only unpublishes remotely, stop is needed on to stop the track locally (make it go to black frames)
                                    publication.track.stop();
                                  }
                                );

//unmute
 createLocalVideoTrack({
                                  width: VIDEO_WIDTH,
                                  height: VIDEO_HEIGHT,
                                })
                                  .then((localVideoTrack: any) => {
                                    return room.current.localParticipant.publishTrack(
                                      localVideoTrack
                                    );
                                  })
                                  .then((publication: any) => {
                                    console.log(
                                      "Successfully unmuted your video:",
                                      publication
                                    );
                                    // setUserVideoEnabled(true);
                                  });

so what i've noticed is this:
for local participant:

• on mute: the trackUnpublished/trackUnsubscribed events aren't called, you can use track.on("stopped", ...) to detect the mute
• on unmute: the trackPublished is called with (participant.on("trackPublished", (publication: any) => { ... }) note that in the callback publication.track EXISTS, however, trackSubscribed will not be called locally

for remote participant:

• on mute: you will get both trackUnpublished and trackUnsubscribed events, so you handle that as you wish
• on unmute: trackPublished is called, however publication.track will be NULL + publication.isSubscribed = false until the track is subscribed to, so you should handle that then.

here are more details about publishing/subscribing details https://www.twilio.com/docs/video/migrating-1x-2x#remotetrackpublication

hope this helps anyone debugging twilio in the future. a note for the maintainers: i think it'd be really helpful if there was a better distinction between publications and tracks. for example, participant.on("trackPublished") returns a publication, not a track. and in the quickstart guide there's a lot of referring to publications as tracks, which is confusing and results in a lot of mistakes like track.stop() when it should be track.track.stop().

edit: also wanted to agree with @dferrazm, the lack of consistent events between local/remote media means writing react components is a lot more tedious since there are a bunch more cases to handle.

[nlukovic]

Another reason why this event is needed. I'm assembling a special name for each user that in turn gets tracked on backend for special compositing of videos. Which is why every user has to have one and only one track. If i call unpublish i don't know when i can publish a new track with the same name because i get duplicated track exception even though i did call unpublish before...

[bijmon]

What's the status of this issue? I have facing an issues because the event is not available. Please help.

[arunpbk-stayqrious]

+1 for this issue, expected trackUnpublished to be fired.

We are using the following workaround for now:

const publication = participant.unpublishTrack(track);
participant.emit('trackUnpublished', publication);

[manjeshbhargav]

Closing this as duplicate of #1007 .

[xdumaine]

Uhh this one existed first and is 2 years old.