Ring: Live Stream Audio

[dgreif]

I know a lot of you are eagerly waiting audio in the live streams, but unfortunately it's not going to be as easy as I originally thought. Ring only appears to support PCMU, while HomeKit only supports Opus, AAC-ELD, AMR and AMR-WB. This means that we need to convert every audio packet from Ring to one of the supported HomeKit codecs, and then convert the other way if we want 2-way audio. While this is definitely possible, it's going to take a lot of research into existing encoders/decoders and finding a way to do it efficiently so that it will work on Raspberry pis (since that's what most homebridge setups run on). This is not a challenge that I'm currently looking to take on myself since live video was my main goal (and a huge challenge in itself). I would love some help from the community on this one, especially if anyone out there is familiar with audio codecs and RTP.

If anyone starts actively working on this, please comment below to avoid duplicate work. For those of you who really want this feature but are unable to help on the code side, please throw a 👍 or ❤️ on the issue, rather than blowing up this thread with +1 comments 😄.

[bourdakos1]

I think the easiest approach would be to just use ffmpeg, any manual encoding/decoding will be a nightmare

[dgreif]

Just wanted to give an update on audio & streaming. Over the weekend I got ffmpeg pulled into the SIP client and used it to do audio transcoding for HomeKit. It works 🚀 🎉! I still have a fair amount of cleanup and testing to do before I will be ready to release the changes, but I should have a functioning version of HomeKit audio later this week. The initial version will just be 1-way audio (haven't tried to do 2-way yet), and I can't promise that the quality will be amazing. Big thanks to @bourdakos1 for pushing me in the right direction with your streaming code!

[EarthRise626]

@dgreif Awesome news!

[dgreif]

Ok, thanks for everyone's patience on audio! v5.6.1 is out and 1-way audio is working! Now, there are some caveats that I need to cover...

• My approach uses ffmpeg to transcode the audio into AAC-eld. This requires you to build a special version of ffmpeg - see instructions in the Ffmpeg wiki. I tried really hard to avoid this, preferring to use opus which works with more common builds of ffmpeg. The problem is that I can't get opus to stream well at all for some reason, but AAC-eld works great. If anyone knows a fair amount about audio codecs, please open a new GitHub issue and I would love to discuss the issue/solutions.
  
  
  
  • Heads up on the ffmpeg build, it takes a while! Especially on raspberry pi. If you really want audio, please be patient and follow the instructions step by step. If you run into issues going through those steps, please google it. I probably won't have any more insight into the issue than you do 😉
  
  
  • Audio takes a while to start after video start playing - about 5 seconds in most of my testing. This is because we have to start the stream, then start ffmpeg, and ffmpeg is taking that long to start passing along the transcoded audio packets 😞. I _might_ be able to eliminate this delay, but it will take a fair amount of work
  
  
  • Audio quality is fair...but maybe not as good as the Ring app. If this is frustrating you, please remember that the intention of homebridge is to _bridge_ the gap until companies offer native HomeKit support. It may not be perfect, but I think I have accomplished that goal at this point. 🤞 native support soon 🤞
  
  
  • 2-way audio is on my todo list, no promises tho
  
  

[ifeign]

Has there been any progress on 2-way audio? Is it even possible?

[dgreif]

@ifeign someone else just asked about this in #237. I'll post any updates for 2-way audio in that issue.