Cognitive-services-speech-sdk: Custom real-time media source as cognitive source

[Anduin2017]

Is your feature request related to a problem? Please describe.
I am working on a project which converts the user's voice to real-time text. It works in browser with JavaScript and the SDK works fine. But I'm frustrated when I try to use a custom media source, (not the microphone) like the voice from a custom running application output or media stream, as the media source. I searched the document but found no result.

Describe the solution you'd like
I want to use a custom media source as a cognitive source. Like a media stream.

Describe alternatives you've considered
I have to save the media stream as a file and use the file as a cognitive source. But that seems not to be a good practice.

Additional context

https://developer.mozilla.org/en-US/docs/Web/API/MIDIAccess

The MediaDevices.getUserMedia() method prompts the user for permission to use a media input which produces a MediaStream with tracks containing the requested types of media. That stream can include, for example, a video track (produced by either a hardware or virtual video source such as a camera, video recording device, screen sharing service, and so forth), an audio track (similarly, produced by a physical or virtual audio source like a microphone, A/D converter, or the like), and possibly other track types.

[amitkumarshukla]

Thanks a lot for raising the issue. We are looking into it and will get back to you soon

[pankopon]

Unfortunately there isn't real good documentation on this right now. I created a backlog item to create a how-to for using push/pull streams.

For a push stream, there are examples in node samples e.g.

• function openPushStream:
  https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/js/node/index.js#L17
  
  
  
  • instead of loading a file, you would get audio from a custom source
  
  
• the stream is used here for recognizing speech once:
  https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/js/node/speech.js#L11

For a pull stream, there's an example in tests:
https://github.com/microsoft/cognitive-services-speech-sdk-js/blob/master/tests/SpeechRecognizerTests.ts#L1133

Could you try adapting code from these examples for your case?

Adding @glharper @rhurey for follow-up.

[oscholz]

@Anduin2017 I am going to close this issue per @pankopon's description above. If you still have trouble with this, please let us know. When we have better documentation for this, or have otherwise resolved the backlog item we created, we will update this issue so you know when any changes from this will show up in the SDK.