Voice: [iOS] Crash on com.apple.coreaudio.avfaudio: required condition is false: IsFormatSampleRateAndChannelCountValid(format)

[jamsch]

Hi, I'm getting an error while starting up speech recognition on an iPad Air 2. Not sure what is causing the issue.

Environment

"device": {
    "family": "iOS",
    "arch": "arm64",
    "storage_size": 63989493760,
    "free_memory": 51675136,
    "memory_size": 2084569088,
    "boot_time": "2017-11-13T07:29:50Z",
    "model": "iPad5,4",
    "usable_memory": 1687126016,
    "type": "device"
},

"os": {
    "rooted": false,
    "kernel_version": "Darwin Kernel Version 17.2.0: Fri Sep 29 18:14:52 PDT 2017; root:xnu-4570.20.62~4/RELEASE_ARM64_T7001",
    "version": "11.1.1",
    "build": "15B150",
    "type": "os",
    "name": "iOS"
}

Exception

Application threw exception com.apple.coreaudio.avfaudio: required condition is false: IsFormatSampleRateAndChannelCountValid(format)

Originated at or in a subcall of folly::Expected<long long, folly::ConversionCode> folly::detail::str_to_integral<long long>(folly::Range<char const*>*)

[jamsch]

Narrowed it down to the following lines. Specifically, it crashes on line 97. Guessing it may have something to do with the recordingFormat variable.

Voice.m:95-101

AVAudioFormat* recordingFormat = [inputNode outputFormatForBus:0];

[inputNode installTapOnBus:0 bufferSize:1024 format:recordingFormat block:^(AVAudioPCMBuffer * _Nonnull buffer, AVAudioTime * _Nonnull when) {
    if (self.recognitionRequest != nil) {
        [self.recognitionRequest appendAudioPCMBuffer:buffer];
    }
}];

[jamsch]

Just got my hands on a brand new iPad (2017) and the same issue occurs on it as well.

"device": {
    "family": "iOS",
    "arch": "arm64",
    "storage_size": 31999954944,
    "free_memory": 41123840,
    "memory_size": 2075656192,
    "boot_time": "2017-11-24T00:07:53Z",
    "model": "iPad6,11",
    "usable_memory": 1944502272,
    "type": "device"
},
"os": {
    "rooted": false,
    "kernel_version": "Darwin Kernel Version 17.2.0: Fri Sep 29 18:14:51 PDT 2017; root:xnu-4570.20.62~4/RELEASE_ARM64_S8000",
    "version": "11.1.2",
    "build": "15B202",
    "type": "os",
    "name": "iOS"
},
{
    "cpu_subtype": 0,
    "name": "/System/Library/Frameworks/AVFoundation.framework/Frameworks/AVFAudio.framework/AVFAudio",
    "revision_version": 0,
    "type": "apple",
    "major_version": 1,
    "image_vmaddr": "0x18658c000",
    "image_addr": "0x189d20000",
    "minor_version": 0,
    "cpu_type": 16777228,
    "image_size": 991232,
    "arch": null,
    "uuid": "7583a123-2e4d-363f-bfb3-fad21d33622d"
},

[mrwordev]

I have the same issue here, It only happen on real device only. This include iPhone5, iPhone6s, iPhone7. No luck on fixing this so far.

[Damnum]

Same issue here, this is really holding us back from going into production

[mrwordev]

@jamsch I have manage to make it work by adding category and a few function call there.
In Voice.m starting from line 38

    NSError* audioSessionError = nil;
    self.audioSession = [AVAudioSession sharedInstance];
    [self.audioSession setCategory:AVAudioSessionCategoryRecord error:&audioSessionError];

    if (audioSessionError != nil) {
        [self sendResult:RCTMakeError([audioSessionError localizedDescription], nil, nil) :nil :nil :nil];
        return;
    }
    [self.audioSession setMode:AVAudioSessionModeMeasurement error:&audioSessionError];
    if (audioSessionError != nil) {
        [self sendResult:RCTMakeError([audioSessionError localizedDescription], nil, nil) :nil :nil :nil];
        return;
    }
    [self.audioSession setActive:YES withOptions:AVAudioSessionSetActiveOptionNotifyOthersOnDeactivation error:&audioSessionError];
    if (audioSessionError != nil) {
        [self sendResult:RCTMakeError([audioSessionError localizedDescription], nil, nil) :nil :nil :nil];
        return;
    }

[jamsch]

@mrwordev
Haven't had the chance to test thoroughly but it did seem to work fine although using the speech recognition while any audio is currently playing (e.g. using react-native-sound) seems to break. I'll be out for the weekend but I'll test it further on Monday.

[jamsch]

So AVAudioSession is a singleton/shared instance for your whole app. Changing the category of a shared audio session to AVAudioSessionCategoryRecord will interrupt any audio that is playing. react-native-sound and react-native-audio-toolkit don't play nicely with this (when AVAudioSessionCategoryPlayback is used) because it seems to either lose the pointer for the audio resource and replace it with the input/microphone source, interrupt the currently playing audio or decreases the volume of the currently playing audio by 80% and all subsequent audio regardless whether speech recognition is active.

react-native-sound has an API option to set the category and this package should too considering that changing it affects other media players. It would then allow for changing the category from Playback to PlayAndRecord when recording, and then back to Playback once done.

Another issue I noticed is that if the startSpeech function were called while speech recognition had already started there's a return but no callback which results in an unresolved promise if I'm correct.

[ohtangza]

@ghsdh3409 Please keep track of this issue.

[zxcal]

In my case, the error occurs when I start recognizing immediately after the destroy function. The time destroy function return the promise is not after it truly destroy the recognition. I use setTimeout to delay the restart to about 500 ms to make sure that the recognition is destroyed completely. This solve my problem.

[dmwallace]

Also getting this issue when including react-native-sound in my project

[ainulya16]

Sound not playing after modify line 38 on Voice.m

Any solution?

[rajkissu]

I'm facing this issue of no sound as well. After adding code on line 38 as per @mrwordev's suggestion, react-native-voice works on iOS - but it kills sound on other modules like react-native-tts

Anyone managed to find a solution?

[jamsch]

I'll be using react-native-voice on a large production app on iOS quite soon with a fork of this with a few slight fixes. https://github.com/jamsch/react-native-voice/ . Will probably open a few pull requests and clean up the code once everything settles down.

The main differences are:

• The audio category is reverted back to the original category set prior to using speech recognition
• Should take bluetooth/wireless sound output devices in to account
• A method for checking whether speech recognition is ready on iOS.
• try/catch blocks around code prone to crashing the app, which instead throws an error on the JavaScript side, which allows the user to rather be notified if speech recognition failed.
• Better error handling/throwing
• Copied setCategory API from react-native-sound in case the user wants to control the audio category.

[ohtangza]

@jamsch Sounds very interesting! I would love to review your PR. :)

[weihuali0509]

This is not working. Any ETA for the PR @jamsch ? Thanks!

[abhishekbhardwaj]

@jamsch Great work man! Would it be possible for you to PR the changes so far?

[jamsch]

The changes up till https://github.com/jamsch/react-native-voice/commit/ca0485de2d467a3339269e2e2039cc91b4438dd8 have been live in a production app the past few days. There's been some significant changes since then, mostly around handling errors.

According to my error tracker, one user has experienced the issue:
NSInternalInconsistencyException: SFSpeechAudioBufferRecognitionRequest cannot be re-used but besides that one error it's been running smoothly.

[abhishekbhardwaj]

@jamsch Have you tried your changes on Android too or just iOS? Cuz it kept giving me 'speech unavailable' on Android.

[jamsch]

@abhishekbhardwaj Should now be fixed, forgot about returning/breaking out of a switch statement.

[abhishekbhardwaj]

@jamsch perfect. Thank you man!

[anaerobic]

@jamsch I am also seeing an error on await Voice.start("en_US") in an Android API 24 simulator using the VoiceTest example and the following in my package.json:

"react-native": "0.54.1",
...
"react-native-voice": "git+https://github.com/jamsch/react-native-voice.git",

and the error in logcat is:

[Info] 04-19 22:38:54.094  3847  3878 E ReactNativeJS: { code: 'not_available',
04-19 22:38:54.094  3847  3878 E ReactNativeJS:   line: 143090,
04-19 22:38:54.094  3847  3878 E ReactNativeJS:   column: 18,
04-19 22:38:54.094  3847  3878 E ReactNativeJS:   sourceURL: 'http://10.0.3.2:8081/index.delta?platform=android&dev=true&minify=false' }

I've tried react-native-link, restarting packager, clearing cache, reinstalling app, etc.

I'll be happy to assist further if needed, thanks much!

[anaerobic]

actually i see now this is thrown when speech recognition is considered to be not available on the device at https://github.com/wenkesj/react-native-voice/compare/master...jamsch:master

I verified that Google Text-to-Speech is installed via Play Store, selected on Settings and that voice data has been downloaded.

[anaerobic]

okay, I found that @jamsch /master works on a connected android device after deleting the android/build/generated+intermediate+tmp directories and rebuilding all of that, so the "not_available" problem might be specific to virtual android devices.

[alvaro1728]

@jamsch Thanks for all the fixes. I'd like to use your fork in my project. How do I npm install it?

[folofse]

@jamsch I found that the "NSInternalInconsistencyException: SFSpeechAudioBufferRecognitionRequest cannot be re-used" -problem might be caused by a bug in react-navigator which seams to mount the views twice. It works fine when all is setup and initialized, but if the function runs for the first time and the user gets the "Allow microphone" - alert, as soon as they click "Allow", the app crashes due to this problem. The error occurs when Voice.start() on this line in Voice.m self.recognitionTask = [self.speechRecognizer recognitionTaskWithRequest:self.recognitionRequest resultHandler:^(SFSpeechRecognitionResult * _Nullable result, NSError * _Nullable error) {. I tried to fix the problem by throttle the Voice.start() calls in the view, but the views seams to get mounted from different instances and they calls componentDidMount at the same time so its a bit tricky to get it to work.

Edited
I manage to fix this problem by adding a check in the beginning of setupAndStartRecognizing to see if the session already had begun.

- (void) setupAndStartRecognizing:(NSString*)localeStr {
    if(self.sessionId){
        return;
    }

[ManhVuTien]

@folofse

• (void) setupAndStartRecognizing:(NSString*)localeStr {
  if(self.sessionId){
  return;
  }

It can't recognize voice

[sibelius]

@jamsch in your fork have you solved your problem ? https://github.com/jamsch/react-native-voice

are you willing to send a PR to fix this?

[sibelius]

I think it could be fixed here https://github.com/wenkesj/react-native-voice/commit/94eed856a223a7296570d24953b9c8d908952c8e

[sibelius]

this is probably fixed on master, based on this https://github.com/wenkesj/react-native-voice/commit/fade2bddbec6be35165e8feee7cfa6967b93c669

can anybody confirm this?

[sibelius]

fixed on master

[zainmusani]

Working on iOS device and Crashing on iOS Simulator

[kingarawana]

On iOS 12 it crashes on device but works on simulator. Tried on iPad Pro 3 and iPhone 6 device.

[tronin]

The same problem with react-native-tts.

[ddgromit]

If anyone is still having this problem, first make sure you're using the master version of this package. Version 0.2.6 that is published on NPM is a year old and doesn't contain the fix.

Link directly to the github in your dependencies:

    "react-native-voice": "wenkesj/react-native-voice",

[monggos]

@ddgromit still having the issue of..

on first init, all just fine until you need to start again to capture the voice for the second time.

is there any solution yet? btw, i am using the master version 0.2.6

thanks!

[rajeevAgilehub]

fixed on master

I copied Voice.m from master and replace the existing code (node_modules/react-native-voice/ios/Voice/Voice.m), it fixed the conflict problem between react-native-voice and react-native-tts 👍🕺🏿

[monggos]

@rajeevAgilehub i did fixed my issue by using this package instead. https://github.com/Evgen74/react-native-voice

i think he fixed it same way as yours?

[lukemcgregor]

Im getting this issue in 0.3.0, is it still not fixed in that version? or is this a regression of something?

[teslavitas]

I am getting this event with the version from wenkesj/react-native-voice . Sometimes after 20-30 cycles of stop/start, sometimes faster.