Twilio-video.js: Wrong Typescript type definitions for RemoteVideoTrack and RemoteAudioTrack ?

[joakimriedel]

• [X ] I have verified that the issue occurs with the latest twilio-video.js release and is not marked as a known issue in the CHANGELOG.md.
• [ X] I reviewed the Common Issues and open GitHub issues and verified that this report represents a potentially new issue.
• [ X] I verified that the Quickstart application works in my environment.
• [ X] I am not sharing any Personally Identifiable Information (PII)
  or sensitive account information (API keys, credentials, etc.) when reporting this issue.

Code to reproduce the issue:

function (track: LocalTrack) {
      if (track instanceof LocalVideoTrack) {
            // works
      }
}
function (track: RemoteTrack) {
      if (track instanceof RemoteVideoTrack) {
            // throws: right-hand side of 'instanceof' is not an object
      }
}
function (track: LocalTrack) {
      if (track instanceof LocalAudioTrack) {
            // works
      }
}
function (track: RemoteTrack) {
      if (track instanceof RemoteAudioTrack) {
            // throws: right-hand side of 'instanceof' is not an object
      }
}

I haven't dug deep enough to understand why the above fails, but it seems to be some kind of mismatch between the Typescript definitions and the actual Javascript files. The RemoteVideoTrack and RemoteAudioTrack seems to both inherit from RemoteMediaTrack in the Javascript files, but inherits from AudioVideoTrack in the Typescript typings.

Looking at _track_ in the debugger it also seems as if __proto__ is in fact RemoteMediaTrack for the offending classes.

I would expect instanceof to work for remote tracks the same way as for local tracks. Could you point out what I'm doing wrong here?

Right now I am doing this hack, but the code would be much cleaner if I could use _instanceof_:

    const isVideoTrack = function (track: Track): track is VideoTrack {
      return track.kind === "video";
    },
    if (isVideoTrack(track)) {
        // "track.attach()" is now available in Typescript without build error
      }

[PikaJoyce]

Thanks for reporting this issue @joakimriedel !

I will investigate this and get back to you with some of my findings, I will see if there are any actions or betterments we can get from this as well.

Best,
Joyce

[PikaJoyce]

Hey there @joakimriedel !

After investigating a bit, it doesn't seem to be an issue with the TypeScript definitions. But because the operator instanceof requires JS objects to run on, it will only work on LocalAudioTrack as well as LocalVideoTrack. The classes for RemoteVideoTrack and RemoteAudioTrack are not exposed by the SDK. Therefore, the issue you're currently facing is a JavaScript runtime error, not a compile time TypeScript error.

Which means using (track.kind === 'video') { ... } is totally fine!

For example, we use track.kind === "video" in our Twilio Video React App. We also use this in our Video Quickstart as well!

I hope this answered any questions you may have!

I will be closing this issue since there are no action items from this issue. However, we can continue to discuss further on this thread if you're having further issues.

Thanks again,
Joyce

[joakimriedel]

I'm not 100% sure about the inner workings on how instanceof determines the inheritance chain, but it seems to me as if you would have exposed the Typescript typing for RemoteMediaTrack I could have done track instanceof RemoteMediaTrack and it would have returned true since that's where the constructor is (seen by looking at __PROTO__ of the js object).

I believe the inheritance chain in the typings should match the JS prototype chain, such as

Remote(Video|Audio)Track -> RemoteMediaTrack -> AudioVideoTrack

but now it is

Remote(Video|Audio)Track -> AudioVideoTrack

which means I cannot do track instanceof RemoteMediaTrack, since RemoteMediaTrack is not defined in Typescript but used internally in JS.

The problem is that if I have a method say

function (track: Track) {
  // cannot detect if this is remote or local by using .kind, have to use instanceof
}

So the solution with track.kind doesn't help me out here.

The RemoteMediaTrack is constructed in JS using the code here: https://github.com/twilio/twilio-video.js/blob/master/lib/media/track/remotemediatrack.js

[PikaJoyce]

Hey @joakimriedel !

The keyword instanceof does not work on Typescript types. It can only be run on Javascript objects/classes. Basically, any TypeScript definition that does not have an exported class (listed here), will compile down to something like : Track instanceof undefined. This will be inclusive of all RemoteTracks.

For example, even though there's the RemoteMediaTrack definition that is declared, you will run into an error.

You should also be able to tell the type of track via the source. For example, localParticipant.tracks will only contain LocalTracks. And remoteParticipant.on('trackSubscribed', () => ...) will only emit RemoteTracks.

With that being said, here are some possible workarounds:

function isLocalVideoTrack(track: Track) {
  return track instanceof LocalVideoTrack;
}
function isRemoteVideoTrack(track: Track) {
  return track.kind === 'video' && !isLocalVideoTrack(track);
}
function isLocalAudioTrack(track: Track) {
  return track instanceof LocalAudioTrack;
}
function isRemoteAudioTrack(track: Track) {
  return track.kind === 'audio' && !isLocalAudioTrack(track);
}

Please let me know if these work, or if you have any further questions!

Best,
Joyce

[joakimriedel]

Thanks @PikaJoyce for the explanation, these workarounds would work!