Audiokit: Ableton Link Integration

[JoshuaBThompson]

AudioKit and Ableton Link

Just writing down a quick list of methods that get called by Ableton Links 'LinkHut' example app to execute a tempo update.
Since Ableton Link is meant to keep different music apps in tempo / sync then updating the AKSequencer / tempo backend would probably be enough to integrate Link with AudioKit.

Integration

Note:

• Need to download Ableton Link / LinkHut from the [LinkKit release tab]
  (https://github.com/Ableton/LinkKit/releases)
• LinkHut is in the examples folder inside the release zip

Here are the steps taken in LinkHut from Ableton:

Below are brief breakdown of steps taken to setup Ableton Link with an audio engine to update tempo

All steps described below are based of the Ableton Link LinkHut example app.

The process of updating the tempo (when another user set's new tempo):

• First audio engine and Ableton Link objects are setup in ViewController -> viewDidLoad
  
  
  
  • _audioEngine = [[AudioEngine alloc] initWithTempo:_bpm];
  
  
  • This is where AVAudioSession shared instance and callbacks get setup
  
  
  • ABLLinkSetSessionTempoCallback(...)
  
  
  • This is where Ableton Link tempo change gets passed along to user callback to update UI
  
  
  • _audioEngine.linkRef, onSessionTempoChanged, (__bridge void *)self);
  
  
  • _linkSettings = [ABLLinkSettingsViewController instance:_audioEngine.linkRef];
  
  
  • [_audioEngine setQuantum:_quanta];
  
  

• Ableton Link will receive the tempo change from the other user (over the network for ex)
• Ableton Link will send an event to the AVAudioSession audio engine route
• User code has a callback that gets called when route event happens
• See 'AudioEngine.m' in the Ableton LinkHut app example:
  
  
  
  • callback in AudioEngine.m -> audioCallback
  
  
  • This is setup in 'setupAudioEngine' method that gets called during init:
  
  
  • This is where _linkData object is setup to connect to shared instance of AVAudioSession properties
  
  
  • 'AURenderCallbackStruct ioRemoteInput;'
  
  
  • gets called to setup i/o callback using 'audioCallback' function
  
  
  • audioCallback is defined to checks for a tempo update and saves the timestamp and new tempo
  
  
  • See 'static OSStatus audioCallback' method in AudioEngine.m
  
  
  • This is where the tempo actually gets updated when we receive an tempo event from Ableton Link
  
  
  • If new tempo is valid it passes along the new tempo event to another callback that informs the UI to display the new tempo to the user
  
  
  • This happens in 'ABLLinkSetSessionTempoCallback' defined in the ViewController.swift
  
  
  • This callback is setup in ViewController.swift -> viewDidLoad -> ABLLinkSetSessionTempoCallback
  
  

[JoshuaBThompson]

Probably just need to add methods in AKSequencer init that takes in Ableton Link Enable / Disable and Callback that user can execute to update tempo values in the View / UI.
Something like:

let sequence = AKSequencer(ablink_en = True, ablink_callback=myCallback);

func myCallback(new_tempo: Int){
   //sequence.tempo was just updated to new_tempo
   self.myTempoLabel.text = new_tempo
}

[JoshuaBThompson]

Just need to have something similar to what is done in (ex App from Ableton) LinkHut -> AudioEngine.m.
See in AudioEngine.m:

• initWithTempo
• initLinkData
• setupAudioEngine
  
  
  
  • audioCallback
  
  

[aure]

Just wanted to respond that I have read up on the Link API and I guess this will be set up a little like Audiobus support, where it is not in AudioKit directly, but instead a file, or set of files, defining extension to AK classes could be dropped into a project.

[aure]

Is there any way that you can share like a skeleton / cartoon version of your project with me privately? I just feel like Ableton Link integration is something that is driven by an application, not a framework. Anything we can do in AudioKit to make Link integration easier would be great, but without the context of an app, it seems like it won't be driven by a real use-case scenario.

[JoshuaBThompson]

Yeah we can show you our app code.
I was also thinking it might be easier just to use the ableton link
'LinkHut' example app and just replace the tempo sound generating portion
of the app with AKSequencer loop.

On Wed, Jun 28, 2017 at 3:02 PM Aurelius Prochazka [e-mail]
wrote:

Is there any way that you can share like a skeleton / cartoon version of
your project with me privately? I just feel like Ableton Link integration
is something that is driven by an application, not a framework. Anything we
can do in AudioKit to make Link integration easier would be great, but
without the context of an app, it seems like it won't be driven by a real
use-case scenario.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/audiokit/AudioKit/issues/937#issuecomment-311805140,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALFpV6peBewvuNYy8KkhsjeCMHee2X3dks5sIs2AgaJpZM4OGzK5
.

[aure]

Yeah, not sure how easy starting with their LinkHut example will be.

[JoshuaBThompson]

I think we should probably use a very simple AK app to demonstrate
integration.
I think we can just throw up something with AKSequence looper as a click
track that lets a user change the tempo.
I can put something together this week.
Then if that works we can test it with our app.
What do you think?

On Wed, Jun 28, 2017 at 3:18 PM Aurelius Prochazka [e-mail]
wrote:

Yeah, not sure how easy starting with their LinkHut example will be.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/audiokit/AudioKit/issues/937#issuecomment-311808144,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALFpV1zIthgJDvAtuB3jF30ZwxhtxDvHks5sItEegaJpZM4OGzK5
.

[aure]

Sounds great to me.

[kvacquier]

That's really interresting. I works on an app using audio kit to analyse microphone & midi input to manage lightning fixtures and we planned to use Ableton Link so that's really interesting .

I believe that sync/control need to be a separated control listener like the midi one that give informations from a listener to let the user do "what he want" with it ;)

[SS-HU]

Hello!
This feature would be awesome. Are there any working examples?
Would be delighted to help!

[ryanfrancesconi]

@dave234 AKTiming extension?

[SS-HU]

Does AKTiming connect with Live?

[ryanfrancesconi]

@SS-HU no, AKTiming is a protocol @dave234 wrote to standardize time sync within AudioKit.

[SS-HU]

That's awesome.
Is it tight enough for production purposes?

[ryanfrancesconi]

The timing is exact, yes.

[ryanfrancesconi]

Check out AKClipPlayer, AKTimeline, AKPlayer and related classes

[SS-HU]

Awesome. Thanks, Ryan! I will!

[dave234]

We still need to test the AKPlayer implementation of AKTiming.

[SS-HU]

Perhaps we could put together a tutorial of some sort for the purpose while testing.

[dave234]

@SS-HU That makes a lot of sense.

[ryanfrancesconi]

@SS-HU AKPlayer is brand new, so that is certainly not yet for production purposes!

[SS-HU]

Awesome, great to know.
Although as long as the timing for AKTiming protocol is tight and stable, putting together a tutorial for Live integration could be done.

[dave234]

Back to Link integration. My idea for this is to create AKBeatTiming that will inherit from AKTiming. It will have similar function as AKTiming, but will specify the facilities necessary for converting seconds to beats (tempo, time signature, beatAtTime, timeAtBeat). Once we've nailed down AKBeatTiming, we can create AKLoopTiming which will inherit from AKBeatTiming. AKLoopTiming will specify the facilities needed to synchronize looping audio. Then finally, we can create something like AKLinkNode. AKLinkNode would call a callback when link state changes, the callback will include the new link state. Where it all comes together: We can create a class that uses AKLinkNode and an array of AKLoopTiming objects to synchronize the playback of all the AKLoopTiming objects. All of the names and protocol/class relationships are subject as the only part implemented is AKTiming, so this is basically a rough draft at this point. Simple, right?

[SS-HU]

Simplicity and time consuming are apart.

[ryanfrancesconi]

Sure, I'll contribute _AKFreeFormJazzExplorationTiming_

You do the rest @dave234 !

[SS-HU]

Would be delighted to contribute time towards the objective.

[dave234]

AKLoopTiming is the tricky part. If we decide that loop start is at zero, it's not so bad. However, if we decide that loopStart can be non-zero, then we need to be able to handle the offset which involves "resetting" the reference time (The playback started time that all other times will be derived from).

[SS-HU]

Would that accommodate timing changes?

[dave234]

That's the hope!

[SS-HU]

Sweet. Considering the expertise, please lead on!

[dave234]

Here's a naive implementation of what AKBeatTiming might look like.
https://github.com/AudioKit/AudioKit/blob/f99e45103f42b02f995df7271264828d9d3e1dc1/Examples/iOS/MetronomeSamplerSync/MetronomeSamplerSync/ViewController.swift#L45
This implementation has loop start locked at zero.

[dave234]

These are the important lines though.

let now = AVAudioTime(hostTime: mach_absolute_time())
metronome1.setBeatCount(beatCount, at: now)
metronome2.setBeatCount(beatCount, at: now)

[SS-HU]

Was considering the metronome to solve the background requirement of CoreMidi in AudioKit, while it appears to render the metronome, and could not find a metronome callback. Maybe I'm doing it wrong, but the live clock is Midi protocol. Would that be performance plausible?

[dave234]

AKSamplerMetronome doesn't have a callback, it just triggers a sampler.

[SS-HU]

Which in turn does not solve the equation of wanting to synchronise with sequencers which might not be sample based, but oscillated and would require triggering.

[aure]

I'm uncomfortable with an issue that has been commented on since December of last year still being here. It seems to me that having Ableton Link integration on the project list is enough. Link integration should come naturally when someone on the core team adds Link to their projects using AudioKit, not from guessing how a project might implement Link. Feel free to comment on this thread, I won't lock it for now, but I will close it as an issue.