Jitsi-meet: On Android SDK 26+ it sets Audio Device to EARPIECE instead of SPEAKER joining the room
Description
When creating the room with Android SDK, the RN app sets the audio device to EARPIECE instead of SPEAKER, but the UI shows SPEAKER as device selected. This only occurs when the RN app it's running on Android SDK 26 or higher (I tested it on Android 8.1 and on Android 9). I tested using Android 6.0.1 and never fails.
When I change the call mode to Only Audio and return to Video Call, the audio is set on device SPEAKERS perfectly.
I also tested using a Bluetooth Audio device before starting the call. It sets the audio to BLUETOOTH device and let you change it to SPEAKERS without problems. It only fails when there only are EARPIECE and SPEAKERS as audio devices and we start a video call.
Logcat:
2019-02-20 18:13:41.473 29977-30120/? D/JitsiConnectionService: startCall UUID=088BEF72-BEC1-4BC1-87E0-444821D8BCA5, h=https://videocall.iontab.es/2ABRxWldxieqSMVLzyo9hz2JARSsqWgo, v=true
2019-02-20 18:13:41.474 29977-30116/? I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'Focus jid set to: undefined'
2019-02-20 18:13:41.475 29977-30116/? I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'Session ID: null machine UID: 5d0f001db1bad31f303e8432d00ec526'
2019-02-20 18:13:41.494 29977-29977/? I/org.webrtc.Logging: SurfaceEglRenderer: : surfaceChanged: format: 4 size: 1074x2154
2019-02-20 18:13:41.495 29977-29977/? I/org.webrtc.Logging: EglRenderer: setLayoutAspectRatio: 0.49860725
2019-02-20 18:13:41.516 1436-6362/? D/PowerManagerService: PARTIAL_WAKE_LOCK 'rohit_bg_wakelock' ACQ=-1s724ms (uid=10344 pid=29977)
2019-02-20 18:13:41.747 29977-29977/? D/JitsiConnectionService: onCreateOutgoingConnection 088BEF72-BEC1-4BC1-87E0-444821D8BCA5
2019-02-20 18:13:41.750 29977-30120/? D/JitsiConnectionService: updateCall: 088BEF72-BEC1-4BC1-87E0-444821D8BCA5 hasVideo: true
2019-02-20 18:13:41.773 29977-30116/? I/ReactNativeJS: '[modules/xmpp/strophe.ping.js]', ': ', 'XMPP pings will be sent every 10000 ms'
2019-02-20 18:13:41.805 1436-1571/? W/BroadcastQueue: Permission Denial: receiving Intent { act=android.intent.action.PHONE_STATE flg=0x1000010 (has extras) } to ProcessRecord{7c117b1 29977:es.ionide.ionVirtualVisit.android/u0a344} (pid=29977, uid=10344) requires android.permission.READ_PRIVILEGED_PHONE_STATE due to sender android (uid 1000)
2019-02-20 18:13:41.811 1436-1571/? W/BroadcastQueue: Permission Denial: receiving Intent { act=android.intent.action.PHONE_STATE flg=0x1000010 (has extras) } to ProcessRecord{7c117b1 29977:es.ionide.ionVirtualVisit.android/u0a344} (pid=29977, uid=10344) requires android.permission.READ_PHONE_STATE due to sender android (uid 1000)
2019-02-20 18:13:41.816 1436-1571/? W/BroadcastQueue: Permission Denial: receiving Intent { act=android.intent.action.PHONE_STATE flg=0x1000010 (has extras) } to ProcessRecord{7c117b1 29977:es.ionide.ionVirtualVisit.android/u0a344} (pid=29977, uid=10344) requires android.permission.READ_PHONE_STATE due to sender android (uid 1000)
2019-02-20 18:13:41.817 29977-29977/? I/TelecomFramework: ConnectionService: notifyCreateConnectionComplete TC@20_1: (...->CSW.hCCC)->CS.crCoC->H.CS.crCoC@E-E-E-E8g
2019-02-20 18:13:41.821 29977-29977/? D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
2019-02-20 18:13:41.822 29977-29977/? I/chatty: uid=10344(es.ionide.ionVirtualVisit.android) identical 1 line
2019-02-20 18:13:41.822 29977-29977/? D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
2019-02-20 18:13:41.841 29977-30116/? I/ReactNativeJS: '[__filename]', '
2019-02-20 18:13:41.870 29977-30116/? I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'Focus jid set to: [email protected]'
2019-02-20 18:13:41.871 29977-30116/? I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'Authentication enabled: false'
2019-02-20 18:13:41.871 29977-30116/? I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'External authentication enabled: false'
2019-02-20 18:13:41.872 29977-30116/? I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'Sip gateway enabled: false'
2019-02-20 18:13:41.892 29977-29977/? D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: SPEAKER, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
2019-02-20 18:13:41.892 29977-30138/? D/AudioMode: Update audio route for mode: 0
2019-02-20 18:13:41.937 29977-30116/? I/ReactNativeJS: '[modules/xmpp/ChatRoom.js]', ': ', 'entered', '[email protected]/focus', { affiliation: 'owner',
role: 'moderator',
jid: '[email protected]/focus7785178195662337',
isFocus: true,
isHiddenDomain: false }
2019-02-20 18:13:41.938 29977-30116/? I/ReactNativeJS: '[modules/xmpp/ChatRoom.js]', ': ', 'Ignore focus: [email protected]/focus, real JID: [email protected]/focus7785178195662337'
2019-02-20 18:13:41.938 29977-30116/? I/ReactNativeJS: '[modules/version/ComponentsVersions.js]', ': ', 'Got xmpp version: Prosody(trunk nightly build 863 (2018-03-18, e88744fa0985),Linux)'
2019-02-20 18:13:41.939 29977-30116/? I/ReactNativeJS: '[modules/version/ComponentsVersions.js]', ': ', 'Got focus version: JiCoFo(1.0.1.0-440,Linux)'
2019-02-20 18:13:41.944 29977-30116/? D/ReactNativeJS: '[JitsiConference.js]', ': ', 'P2P? isModerator: false, peerCount: 0, hasBotPeer: false => false'
2019-02-20 18:13:41.944 29977-30116/? I/ReactNativeJS: '[modules/xmpp/ChatRoom.js]', ': ', '(TIME) MUC joined:t', 1550682821944
2019-02-20 18:13:41.963 29977-29977/? D/JMActivity: JitsiMeetViewListener CONFERENCE_JOINED {url=https://videocall.iontab.es/2ABRxWldxieqSMVLzyo9hz2JARSsqWgo?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb29tIjoiMkFCUnhXbGR4aWVxU01WTHp5bzloejJKQVJTc3FXZ28iLCJpYXQiOjE1NTA2ODI4MjAsImV4cCI6MTU1MDc2OTIyMCwiYXVkIjoiKiIsImlzcyI6InJlbGF0aXZlcyIsInN1YiI6InZpZGVvY2FsbC5pb250YWIuZXMiLCJqdGkiOiJlYzYwZmRkZGI1ZmRjNGI1MTQ3ODIzZDdlYWI5YTBjNiJ9.jJbPfoJZLhd2AI_HywRKjBB8G5atIP6poGS82K5CHzc}
2019-02-20 18:13:41.963 29977-29977/? D/JMActivity: It joined to the conference.
2019-02-20 18:13:41.963 29977-29977/? I/JMActivity: Videocall initialized
2019-02-20 18:13:41.968 29977-30120/? D/JitsiConnectionService: reportConnectedOutgoingCall 088BEF72-BEC1-4BC1-87E0-444821D8BCA5
2019-02-20 18:13:41.968 29977-30120/? D/JitsiConnectionService: onStateChanged: ACTIVE 088BEF72-BEC1-4BC1-87E0-444821D8BCA5
2019-02-20 18:13:41.969 29977-30138/? D/AudioMode: Update audio route for mode: 2
2019-02-20 18:13:41.969 29977-30138/? D/AudioMode: Selected audio device: SPEAKER
2019-02-20 18:13:41.973 29977-30116/? I/ReactNativeJS: '[modules/xmpp/ChatRoom.js]', ': ', 'Subject is changed to '
2019-02-20 18:13:42.060 29977-30116/? D/ReactNativeJS: '[JitsiConference.js]', ': ', 'P2P? isModerator: true, peerCount: 0, hasBotPeer: false => false'
2019-02-20 18:13:42.074 29977-30116/? I/ReactNativeJS: '[modules/xmpp/ChatRoom.js]', ': ', 'Ignore focus: [email protected]/focus, real JID: [email protected]/focus7785178195662337'
2019-02-20 18:13:42.912 29977-29977/? D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
2019-02-20 18:13:42.913 29977-30138/? D/AudioMode: Update audio route for mode: 2
2019-02-20 18:13:43.084 29977-2763/? I/org.webrtc.Logging: CameraStatistics: Camera fps: 27.
2019-02-20 18:13:44.721 29977-2765/? I/org.webrtc.Logging: EglRenderer: Duration: 4004 ms. Frames received: 103. Dropped: 0. Rendered: 103. Render fps: 25,7. Average render time: 5923 渭s. Average swapBuffer time: 2778 渭s.
2019-02-20 18:13:45.086 29977-2763/? I/org.webrtc.Logging: CameraStatistics: Camera fps: 30.
2019-02-20 18:13:47.089 29977-2763/? I/org.webrtc.Logging: CameraStatistics: Camera fps: 31.
2019-02-20 18:13:48.722 29977-2765/? I/org.webrtc.Logging: EglRenderer: Duration: 4000 ms. Frames received: 120. Dropped: 0. Rendered: 120. Render fps: 30,0. Average render time: 4660 渭s. Average swapBuffer time: 3032 渭s.
2019-02-20 18:13:49.089 29977-2763/? I/org.webrtc.Logging: CameraStatistics: Camera fps: 30.
2019-02-20 18:13:51.089 29977-2763/? I/org.webrtc.Logging: CameraStatistics: Camera fps: 30.
2019-02-20 18:13:51.373 29977-30116/? I/ReactNativeJS: '[modules/rttmonitor/rttmonitor.js]', ': ', 'Stopping RttMonitor.'
2019-02-20 18:13:51.375 29977-30116/? I/ReactNativeJS: '[modules/e2eping/e2eping.js]', ': ', 'Stopping e2eping'
2019-02-20 18:13:51.379 29977-30116/? I/ReactNativeJS: '[modules/xmpp/ChatRoom.js]', ': ', 'do leave', '[email protected]/85ca2a89'
2019-02-20 18:13:51.417 29977-29977/? I/org.webrtc.Logging: SurfaceEglRenderer: : surfaceChanged: format: 4 size: 1074x2028
2019-02-20 18:13:51.419 29977-29977/? I/org.webrtc.Logging: EglRenderer: setLayoutAspectRatio: 0.5295858
Current behavior
It sets the audio device to EARPIECE instead of SPEAKERS on Android 8.1 or higher
Expected Behavior
It should set the audio device to SPEAKERS
Possible Solution
Reading the code I can suspect that maybe, with the ConnectionService new funcionality, there is a problem setting the SPEAKERS route when the audio devices SPEAKERS is set the first time it call 'setMode'.
Steps to reproduce
Running JitsiMeet SDK on Android 8.1 (or 9) and calling loadUrl(). The UI will show that the current devices is SPEAKERS but the audio will come out from EARPIECE device.
Environment details
JitsiMeet SDK - master/7c911eca96a6034bc96276e0a491774df52bc2cd
Android SDK 26 or higher
Phone devices tested: Nokia 7 Plus, Samsung Galaxy S8, Xiaomi Mi A2
JitsiMeet SDK running on Adobe Cordova App
champunes
All 15 comments
Thanks for detailed report ! We'll take a look at this issue soon
paweldomas
on 20 Feb 2019
I have tried it one One Plus 5 and Samsung Galaxy S7 and it worked fine on both. From your logs it looks like the system did not listen to the device change request at:
2019-02-20 18:13:41.969 29977-30138/? D/AudioMode: Selected audio device: SPEAKER
and came back with:
2019-02-20 18:13:42.912 29977-29977/? D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []].
Then the code which selects the device does nothing, because it thinks that the SPEAKER is set.
2019-02-20 18:13:42.913 29977-30138/? D/AudioMode: Update audio route for mode: 2
We could change and reset the 'selectedDevice' if system comes back with device different than the once currently selected, but I'm a bit afraid this will create infinite loop. I need to think more about a proper fix.
paweldomas
on 21 Feb 2019
I've been wondering if the fact the the SDK runs on Cordova makes any difference. Do you have any demo project I could try to reproduce it with ?
paweldomas
on 7 Mar 2019
I haven't been able to reproduce this either. Device: PocoPhone running Lineage OS 16.0 (Android Pie).
saghul
on 7 Mar 2019
I've been wondering if the fact the the SDK runs on Cordova makes any difference. Do you have any demo project I could try to reproduce it with ?
Yes, our App is in production (Google Play) now (https://play.google.com/store/apps/details?id=es.ionide.ionVirtualVisit.android). The problem is that the app only can make videocalls with other or our apps, not between the same app in different devices.
I'm working right now on supporting ConnectionService on my app for using the Phone app for receiving videocalls. I will make more tests, maybe I can see exactly where is the problem.
champunes
on 8 Mar 2019
@champunes What SDK version / commit are you using?
saghul
on 8 Mar 2019
Also, do you do any audio device management of any kind in your app?
saghul
on 8 Mar 2019
I've been doing tests with a Nexus 5 using sdk 23 and a Nokia 7 plus using sdk 28. As ConnectionService is disabled for sdk 25 or less, the audio output works nicely on the Nexus 5. But, on the Nokia 7 Plus the audio output always start being throught earpiece.
The last commit of the clone I made is: 7c911eca96a6034bc96276e0a491774df52bc2cd
champunes
on 8 Mar 2019
Hum, that's weird. I'm out of ideas to be honest.
saghul
on 8 Mar 2019
After I finished some tasks of my app I'm doing tests with the EARPIECE problem. Something I notice is that after the sound negotiation:
D/JitsiConnectionService: onCreateOutgoingConnection 4E51CB67-6A75-4506-A79E-D3F58FC3935A
D/JitsiConnectionService: updateCall: 4E51CB67-6A75-4506-A79E-D3F58FC3935A hasVideo: true
I/ReactNativeJS: '[modules/xmpp/strophe.ping.js]', ': ', 'XMPP pings will be sent every 10000 ms'
I/TelecomFramework: ConnectionService: notifyCreateConnectionComplete TC@4_1: (...->CSW.hCCC)->CS.crCoC->H.CS.crCoC@E-E-E-EAE
D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: SPEAKER, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
D/AudioMode: Update audio route for mode: 0
I/ReactNativeJS: '[__filename]', '
I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'Focus jid set to: [email protected]'
I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'Authentication enabled: false'
I/ReactNativeJS: '[modules/xmpp/moderator.js]', ': ', 'External authentication enabled: false'
There is an extra Audio State Change:
I/org.webrtc.Logging: WebRtcAudioTrack: nativeOutputSampleRate: 48000
I/org.webrtc.Logging: WebRtcAudioTrack: AudioTrack: session ID: 3177, channels: 1, sample rate: 48000, max gain: 1.0
WebRtcAudioTrack: AudioTrack: buffer size in frames: 3844
WebRtcAudioTrack: AudioTrack: buffer capacity in frames: 3844
I/org.webrtc.Logging: WebRtcAudioTrack: startPlayout
I/org.webrtc.Logging: WebRtcAudioTrack: AudioTrackThread@[name=AudioTrackJavaThread, id=1908]
I/ReactNativeJS: '[modules/xmpp/JingleSessionPC.js]', ': ', '(TIME) ICE checking P2P? false:t', 1552305949163
D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
D/AudioMode: Update audio route for mode: 2
I/org.webrtc.Logging: NetworkMonitor: Start monitoring with native observer -1104384256
W/org.webrtc.Logging: NetworkMonitorAutoDetect: Unable to obtain permission to request a cellular network.
I/org.webrtc.Logging: SurfaceEglRenderer: : Reporting frame resolution changed to 720x1280 with rotation 0
I/org.webrtc.Logging: HardwareVideoEncoder: initEncode: 720 x 1280. @ 300kbps. Fps: 60 Use surface mode: true
D/MessengerImpl:
I/org.webrtc.Logging: GlShader: Deleting shader.
I'm doing more tests so I can unterstand why is this happening.
champunes
on 11 Mar 2019
I'm reading "updateAudioRoute" in AudioModelModule.java
Theoretically, after receiving an "onCallAudioStateChanged" with EARPIECE as route, due to the call not being an AUDIO_CALL, the "updateAudioRoute" function should do nothing.
But, if it's the SO making that audio change, Do not "updateAudioRoute" should force a change to SPEAKERS route again?
champunes
on 11 Mar 2019
Theoretically, after receiving an "onCallAudioStateChanged" with EARPIECE as route, due to the call not being an AUDIO_CALL, the "updateAudioRoute" function should do nothing.
No, in that case it should override it and pick the speaker. Note how the user selection is discarded, so it will pick the default for the mode: https://github.com/jitsi/jitsi-meet/blob/81d4f694b710672456a348b4391ce23804b832e5/android/sdk/src/main/java/org/jitsi/meet/sdk/AudioModeModule.java#L458 and https://github.com/jitsi/jitsi-meet/blob/81d4f694b710672456a348b4391ce23804b832e5/android/sdk/src/main/java/org/jitsi/meet/sdk/AudioModeModule.java#L721
But @paweldomas might be a bit more familiar with that part of the code.
saghul
on 11 Mar 2019
I commented this line:
https://github.com/jitsi/jitsi-meet/blob/81d4f694b710672456a348b4391ce23804b832e5/android/sdk/src/main/java/org/jitsi/meet/sdk/AudioModeModule.java#L733
And it works. But the real problem is why is the system provoking an EARPIECE Audio Change?
Theoretically, after receiving an "onCallAudioStateChanged" with EARPIECE as route, due to the call not being an AUDIO_CALL, the "updateAudioRoute" function should do nothing.
No, in that case it should override it and pick the speaker. Note how the user selection is discarded, so it will pick the default for the mode:
jitsi-meet/android/sdk/src/main/java/org/jitsi/meet/sdk/AudioModeModule.java
Line 458 in 81d4f69
userSelectedDevice = null;
and
jitsi-meet/android/sdk/src/main/java/org/jitsi/meet/sdk/AudioModeModule.javaLine 721 in 81d4f69
audioDevice = DEVICE_SPEAKER;
But @paweldomas might be a bit more familiar with that part of the code.
What is happening is that "audioDevice" is setted to DEVICE_SPEAKER. Even with "userSelectedDevice" setted to null, "selectedDevice" is still setted to the last device (DEVICE_SPEAKER). So:
https://github.com/jitsi/jitsi-meet/blob/81d4f694b710672456a348b4391ce23804b832e5/android/sdk/src/main/java/org/jitsi/meet/sdk/AudioModeModule.java#L732
Returns true.
Maybe the solution could be setting "selectedDevice" also to null if the user selection has to be reseted?
Edit:
I've tried setting selectedDevice to null when the user selection has to be reseted. It works nicely (Jitsi SDK set the audio device to SPEAKER again). Should I be worried by that strange Audio Status Change? I think that it is produced after 2 participants exchanges medias (a second one enters the room).
I know that when there are only 2 participants, and they are in the same network, the Jitsi server changes the connection to be point-to-point between the participants. I'm seeing WebRTC Audio activity just before the audio changes event (the following logs are from a Jitsi SDK with "selectedDevice" setted to null when reseting user selection):
I/org.webrtc.Logging: WebRtcAudioTrack: AudioTrack.getMinBufferSize: 7688
I/org.webrtc.Logging: WebRtcAudioTrack: createAudioTrackOnLollipopOrHigher
WebRtcAudioTrack: nativeOutputSampleRate: 48000
I/org.webrtc.Logging: WebRtcAudioTrack: AudioTrack: session ID: 4145, channels: 1, sample rate: 48000, max gain: 1.0
WebRtcAudioTrack: AudioTrack: buffer size in frames: 3844
I/org.webrtc.Logging: WebRtcAudioTrack: AudioTrack: buffer capacity in frames: 3844
I/org.webrtc.Logging: WebRtcAudioTrack: startPlayout
I/org.webrtc.Logging: CameraStatistics: Camera fps: 27.
I/org.webrtc.Logging: WebRtcAudioTrack: AudioTrackThread@[name=AudioTrackJavaThread, id=2073]
I/ReactNativeJS: '[modules/xmpp/JingleSessionPC.js]', ': ', '(TIME) ICE checking P2P? false:\t', 1552314537653
I/org.webrtc.Logging: NetworkMonitor: Start monitoring with native observer -954715040
W/org.webrtc.Logging: NetworkMonitorAutoDetect: Unable to obtain permission to request a cellular network.
D/MessengerImpl:
I/org.webrtc.Logging: NetworkMonitorAutoDetect: Network becomes available: 113
D/JitsiConnectionService: onCallAudioStateChanged: [AudioState isMuted: false, route: EARPIECE, supportedRouteMask: EARPIECE, SPEAKER, activeBluetoothDevice: [null], supportedBluetoothDevices: []]
D/AudioMode: Update audio route for mode: 2
Selected audio device: SPEAKER
champunes
on 11 Mar 2019
@champunes Thanks for digging into this! I think you analysis is correct. Whatever the cause may be, we should always enforce the user selection / default when the system changes the value on its own.
Could you send a PR with your fix please?
saghul
on 11 Mar 2019
Fixed in #3980 thank you!
saghul
on 12 Mar 2019
Related issues
TechnologyClassroom
路
3Comments
eceforge
路
3Comments
jidanni
路
3Comments
wilddylan
路
4Comments
rcidt
路
4Comments
Most helpful comment
@champunes Thanks for digging into this! I think you analysis is correct. Whatever the cause may be, we should always enforce the user selection / default when the system changes the value on its own.
Could you send a PR with your fix please?