AudioKit for iOS: Frequency Discrepancy on Simulator vs Device

[amcinnis]

I am using AudioKit to monitor frequency for a simple guitar tuner application and am experiencing discrepancies in frequency after updating from AudioKit ~4.2 to 4.4, Xcode 9.x to 10, and iOS 11 to 12. Before the updates, I was achieving correct frequency readings on my device. After updating, I am getting accurate results for a low E1 (82.4 Hz) on the simulator, but false readings on the device (alternates from ~23 to ~47 kHz).

I have tried using another device, but achieve the same results.

My viewDidLoad() setting up AudioKit is relatively simple, and I used the AudioKit playgrounds as a guideline:

override func viewDidLoad() {
    super.viewDidLoad()

    // Enable microphone tracking.
    AKSettings.audioInputEnabled = true
    let mic = AKMicrophone()
    let tracker = AKFrequencyTracker(mic)
    let silence = AKBooster(tracker, gain: 0)
    AudioKit.output = silence
    do {
        try AudioKit.start()
    }
    catch {
        print("AudioKit did not start!")
    }
    mic.start()
    tracker.start()

    // Track input frequency, 100ms intervals
    timer = Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) {
        [weak self] (timer) in
        guard let this = self else { return }

        this.frequencyLabel.text = String(format: "Frequency: %.3f Hz", tracker.frequency)
        this.frequencyLabel.sizeToFit()
    }
}

As a sidenote, I am getting Objective-C console output regarding AudioKit classes being implemented in two places. Would this contribute to the issue?

objc[517]: Class AKRhodesPianoAudioUnit is implemented in both /private/var/containers/Bundle/Application/5A294050-2DB2-45C9-BB0A-3A0DE25E87C6/Tuner.app/Frameworks/AudioKitUI.framework/AudioKitUI (0x1058413f0) and /var/containers/Bundle/Application/5A294050-2DB2-45C9-BB0A-3A0DE25E87C6/Tuner.app/Tuner (0x104e177e8). One of the two will be used. Which one is undefined.

Any ideas? Thanks in advance!

[francdn]

I am having a similar problem. On device the frequency is not being detected properly (it goes up and down) and on the simulator I get a crash when I initialize the microphone (AKMicrophone).

This is the error returned by the crash:

'com.apple.coreaudio.avfaudio', reason: 'required condition is false: format.sampleRate == hwFormat.sampleRate'

I am using pretty much the same code you posted but in objective c. It was working fine with the previous versions of Audiokit and Xcode 9.X.

I am also having the warning you mentioned for "AudioKit classes being implemented in two places".

[francdn]

A part from that it seems that AKFrequencyTracker is returning values of between 20000 Hz and 46000 HZ in most of the readings. I am having the same result with an iphone and ipad and different IOS versions.

[francdn]

Btw, the example MicrophoneAnalysis is also crashing in the simulator and device. And the warning "AudioKit classes being implemented in two places" is also there.

2018-09-26 14:15:11.443412-0400 MicrophoneAnalysis[6046:537331] libMobileGestalt MobileGestalt.c:890: MGIsDeviceOneOfType is not supported on this platform.
2018-09-26 14:15:11.939325-0400 MicrophoneAnalysis[6046:537331] [avae] AVAEInternal.h:70:_AVAE_Check: required condition is false: [AVAudioEngine.mm:686:ConnectMultipleOutputs: ([_nodes containsObject: connPoint.node])]
2018-09-26 14:15:11.951759-0400 MicrophoneAnalysis[6046:537331] * Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: [_nodes containsObject: connPoint.node]'
* First throw call stack:
(
0 CoreFoundation 0x0000000103cab29b __exceptionPreprocess + 331
1 libobjc.A.dylib 0x00000001026a6735 objc_exception_throw + 48
2 CoreFoundation 0x0000000103cab022 +[NSException raise:format:arguments:] + 98
3 AVFAudio 0x000000010783c89c _Z19AVAE_RaiseExceptionP8NSStringz + 156
4 AVFAudio 0x000000010787995b _Z11_AVAE_CheckPKciS0_S0_b + 323
5 AVFAudio 0x00000001078be31e _ZN17AVAudioEngineImpl22ConnectMultipleOutputsEP11AVAudioNodeP7NSArraymP13AVAudioFormat + 540
6 AVFAudio 0x00000001078be0ba -[AVAudioEngine connect:toConnectionPoints:fromBus:format:] + 94
7 MicrophoneAnalysis 0x0000000100fdfcd9 $S8AudioKitAAC7connect_2to7fromBus6formatySo11AVAudioNodeC_SaySo0H15ConnectionPointCGSiSo0H6FormatCSgtFZTf4nnnnd_n + 1945
8 MicrophoneAnalysis 0x0000000100faf528 $S8AudioKit8AKOutputPAAE7connect2to3busAA7AKInput_pAaG_p_SitFAA18AKPeriodicFunctionC_Tg5Tf4nnn_g + 616
9 MicrophoneAnalysis 0x0000000100fb05f9 $S8AudioKit8AKOutputPAAE7connect2to3busAA7AKInput_pAaG_p_SitFAA6AKNodeC_Tg5Tf4nnn_g + 9
10 MicrophoneAnalysis 0x0000000100f8b135 $S8AudioKitAAC6outputAA6AKNodeCSgvWZTf4dd_n + 1653
11 MicrophoneAnalysis 0x0000000100f8a092 $S8AudioKitAAC6outputAA6AKNodeCSgvsZ + 66
12 MicrophoneAnalysis 0x0000000100f59ad6 $S18MicrophoneAnalysis14ViewControllerC13viewDidAppearyySbF + 342
13 MicrophoneAnalysis 0x0000000100f59f53 $S18MicrophoneAnalysis14ViewControllerC13viewDidAppearyySbFTo + 51
14 UIKitCore 0x000000010e3707e4 -[UIViewController _setViewAppearState:isAnimating:] + 947
15 UIKitCore 0x000000010e37351d __64-[UIViewController viewDidMoveToWindow:shouldAppearOrDisappear:]_block_invoke + 42
16 UIKitCore 0x000000010e371882 -[UIViewController _executeAfterAppearanceBlock] + 78
17 UIKitCore 0x000000010dc47aba _runAfterCACommitDeferredBlocks + 634
18 UIKitCore 0x000000010dc36424 _cleanUpAfterCAFlushAndRunDeferredBlocks + 384
19 UIKitCore 0x000000010dc56d5a __34-[UIApplication _firstCommitBlock]_block_invoke_2 + 153
20 CoreFoundation 0x0000000103c0ea3c __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
21 CoreFoundation 0x0000000103c0e1f0 __CFRunLoopDoBlocks + 336
22 CoreFoundation 0x0000000103c08a64 __CFRunLoopRun + 1284
23 CoreFoundation 0x0000000103c08221 CFRunLoopRunSpecific + 625
24 GraphicsServices 0x0000000108ffd1dd GSEventRunModal + 62
25 UIKitCore 0x000000010dc3c115 UIApplicationMain + 140
26 MicrophoneAnalysis 0x0000000100f5d634 main + 68
27 libdyld.dylib 0x0000000105079551 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException

[aure]

This will be resolved AudioKit 4.4.1

[lucaji]

I am trying with the develop branch and got the same problems: frequency discrepancy from the AKTracker and a crash on the simulator when initialising the microphone as below quote from @francdn

I am having a similar problem. On device the frequency is not being detected properly (it goes up and down) and on the simulator I get a crash when I initialize the microphone (AKMicrophone).

This is the error returned by the crash:

'com.apple.coreaudio.avfaudio', reason: 'required condition is false: format.sampleRate == hwFormat.sampleRate'

I edited AKMicrophone.swift and changed the initialisation frequencies from 48KHZ to 44.1

        {
        desiredFS = 44100.0
        AKSettings.sampleRate = 44100.0
    }

and now it runs on the simulator without crashing and apparently the frequency is correct.

Still having troubles with a real device though.

[andorfermichael]

I also got the message of duplicate implementation. Furthermore, my app is crashing.

[aure]

@eljeff So, I think this change Jeff made for iPhones with higher sample rates is the cause, I haven't yet changed it in AudioKit because I want him to have a second look.

[francdn]

This will be resolved AudioKit 4.4.1

I am not sure if you were referring to 4.5.1 instead of 4.4.1 but 4.5.0 still has the same issues I mentioned before except for the "duplicate class" message.

[aure]

As I mentioned I tagged Jeff to have a look at this, but if you want to make a pull request @francdn, by all means do so.

[marcosgriselli]

I can take a look at it with Jeff @aure so you can assign it to me.

[marcosgriselli]

@francdn

A part from that it seems that AKFrequencyTracker is returning values of 
between 20000 Hz and 46000 HZ in most of the readings. I am having the
same result with an iphone and ipad and different IOS versions.

Did you work around this issue? I'm having the same problem with physical devices.

[francdn]

@francdn

A part from that it seems that AKFrequencyTracker is returning values of 
between 20000 Hz and 46000 HZ in most of the readings. I am having the
same result with an iphone and ipad and different IOS versions.

Did you work around this issue? I'm having the same problem with physical devices.

Nope. I am waiting until the new version is released with all this fixed.

[aure]

I believe this is fixed in AudioKit 4.5.1. Please verify and reopen the issue if it is not.

[amcinnis]

Issue has been fixed for me. Thank you all for your hard work!

[andorfermichael]

From my point of view, this issue still exists, while the issue was fixed for the simulators I still get the error on real hardware devices which in my case is the iPhone X.

[eljeff]

Good news this should be fixed in the forthcoming version of AK

[aure]

@andorfermichael Which version are you using, it should be working well in 4.5.2, I'm looking at my iPhoneX running the microphone analysis example right now. Make sure to clean out your derived data just in case Xcode is finding an old version of AK. If Xcode can find a wrong version and a new working one, it will use the wrong one.

[andorfermichael]

@aure well, you are right, cleaning out the derived data solved it, thank you.

[joulupukki]

Wondering if this is a problem in 4.9? Running in a simulator the frequency of 220 in the simulator (from my guitar's A string), is showing up as a frequency of 100 when running on my iPhone 11 Pro.

[gavinarrowsmith]

I'm getting the same issue with 4.9.3 running on iPhone XS running iOS 13.3.1.

'A' note returns 220 in simulator, device returns 100.

Is there a solution to this?

[eljeff]

I wouldn't be surprised if this isnt related to a lot of the microphone bugs that 4.9.3 re-introduced.

If you want, check the version of akmicrophone from 4.9.2 and try using that code instead of the new microphone code. You can either just replace akmicrophone w/ the old code, or make a new class and use that using the old akmicrophone code.

[gavinarrowsmith]

I've just checked with 4.9.2 and the issue exists there too.

I've also checked with an iPad to make sure it wasn't something to do with the device.

I've double checked the data that's being reported against actual frequency.

An 'A' at 110 is being reported as 100
An 'A' at 220 is being reported as 201
An 'A' at 440 is being reported as 402

This looks like a 10% reduction. I can multiply the reported frequency by 1.1 and the problem goes away. Hope this helps you find the issue.

[eljeff]

What if you multiply the frequency by 48000/44100?
That could very well explain whats happening, and would be related to a lot of this mic stuff.

[eljeff]

@classicaldude I was thinking the same thing. ~1.08843537414966

[gavinarrowsmith]

Yes, that's exactly the ratio. I've tried to set the sample rate on AKSettings:

AKSettings.sampleRate = 44100

Before I create the MKMicrophoneTracker, but that doesn't have any effect. Is there anywhere else that I should be setting the sampleRate?

It is strange that this only happens on device.

[ClassicalDude]

Sorry, I deleted my previous comment because @eljeff beat me to it.

I have the same issue with my app, that does both fft analysis and has a tuner functionality. I basically had to create switches for everything, for the case of 44.1K (devices from 2017 and earlier) and 48K (devices from 2018). It doesn't seem like anything can override the default rate of the microphone.

EDIT: Although there are many times differences between physical devices and the simulator, it does occasionally help testing when you change the sample rate on the Mac's MIDI Setup app. This way you can test both cases. You'll discover, for instance, that some of AudioKit's features, such as the EZAudioPlot, crash the app when connected to some of the nodes (such as a limiter or a booster) on a 48K device.

[eljeff]

A lot of ak is hardcoded to 44.1, so issues inevitably arise. Definitely a big candidate for improvement, if anyone wants to tackle that.

[gavinarrowsmith]

So, my solution for anyone else with the same problem is:

let frequencyMultiplier: Double = 1.0
let sampleRate = AVAudioSession.sharedInstance().sampleRate
If sampleRate == 48000 {
            frequencyMultiplier = 48000 / 44100
}

...
...

let frequency = microphone.frequency * frequencyMultiplier