Surge: VST3 Linux Inconsistently Working

[baconpaul]

Just here at the top of the thread

As of sep 2019

• builds and runs in Steinberg editor host and reaper on linux
• ui shows up as black in bitwig - the idle loop isn’t getting to the draw loop

[baconpaul]

Forever ago in #128 @asimilon told us the bundle needs to look like vst3sdk/doc/vstinterfaces/vst3loc.html on Linux so that's probably the issue here.

[baconpaul]

Hey OK that @asimilon document is right; we need to package it up like a mac bundle. When I do that I can get bitwig to recognize a VST3 then crash when it loads the plugin. But that's OK. Basically we can make a post-build script to build the .vst bundle.

[baconpaul]

OK great so if you build the steinberg editor host it can also load the .vst3 and gets a core dump in (of all places) xcb_generate_id from VSTGUI::ChildWindow. So clearly something amiss but also clearly our VST3 is now starting to work.

If you want to try this here's a set of commands

# first build and localy install surge vst3 using build-linux

mkdir VS
cd VS
git clone --recursive https://github.com/steinbergmedia/vst3sdk.git
mkdir build
cd build/
sudo apt-get install libsqlite3-dev
cmake ../vst3sdk/
cd public.sdk/samples/vst-hosting/editorhost
make
../../../../bin/Debug/editorhost ~/.vst3/Surge.vst3

and you'll get the loading starting, a window almost popping up, and this core trace:

#0  0x00007ffff6ace736 in xcb_generate_id () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
#1  0x00007ffff54a094e in VSTGUI::X11::ChildWindow::ChildWindow(unsigned long, VSTGUI::CPoint) ()
   from /home/paul/.vst3/Surge.vst3/Contents/x86_64-linux/Surge.so
#2  0x00007ffff549c249 in VSTGUI::X11::Frame::Frame(VSTGUI::IPlatformFrameCallback*, VSTGUI::CRect const&, unsigned int, VSTGUI::IPlatformFrameConfig*) () from /home/paul/.vst3/Surge.vst3/Contents/x86_64-linux/Surge.so
#3  0x00007ffff549c77f in VSTGUI::IPlatformFrame::createPlatformFrame(VSTGUI::IPlatformFrameCallback*, VSTGUI::CRect const&, void*, VSTGUI::PlatformType, VSTGUI::IPlatformFrameConfig*) () from /home/paul/.vst3/Surge.vst3/Contents/x86_64-linux/Surge.so
#4  0x00007ffff545387d in VSTGUI::CFrame::open(void*, VSTGUI::PlatformType, VSTGUI::IPlatformFrameConfig*) ()
   from /home/paul/.vst3/Surge.vst3/Contents/x86_64-linux/Surge.so
#5  0x00007ffff538feed in SurgeGUIEditor::open(void*, VSTGUI::PlatformType const&) ()
   from /home/paul/.vst3/Surge.vst3/Contents/x86_64-linux/Surge.so
#6  0x00007ffff548e3a9 in Steinberg::Vst::VSTGUIEditor::attached(void*, char const*) ()
   from /home/paul/.vst3/Surge.vst3/Contents/x86_64-linux/Surge.so
#7  0x0000555555578771 in Steinberg::Vst::EditorHost::WindowController::onShow (this=0x5555559f5db0, w=...)
    at /home/paul/VS/vst3sdk/public.sdk/samples/vst-hosting/editorhost/source/editorhost.cpp:296
#8  0x000055555558bf72 in Steinberg::Vst::EditorHost::X11Window::Impl::handleMainWindowEvent (this=0x5555559f61d0, event=...)
    at /home/paul/VS/vst3sdk/public.sdk/samples/vst-hosting/editorhost/source/platform/linux/window.cpp:831
#9  0x000055555558ad62 in Steinberg::Vst::EditorHost::X11Window::Impl::<lambda(const XEvent&)>::operator()(const XEvent &) const (
    __closure=0x5555559f6440, e=...)
    at /home/paul/VS/vst3sdk/public.sdk/samples/vst-hosting/editorhost/source/platform/linux/window.cpp:632
#10 0x000055555558ccb9 in std::_Function_handler<bool(const _XEvent&), Steinberg::Vst::EditorHost::X11Window::Impl::init(const string&, Steinberg::Vst::EditorHost::Size, bool, const WindowControllerPtr&, Display*, const WindowClosedFunc&)::<lambda(const XEvent&)> >::_M_invoke(const std::_Any_data &, const _XEvent &) (__functor=..., __args#0=...) at /usr/include/c++/7/bits/std_function.h:302
#11 0x0000555555584167 in std::function<bool (_XEvent const&)>::operator()(_XEvent const&) const (this=0x5555559f6440, 
    __args#0=...) at /usr/include/c++/7/bits/std_function.h:706
#12 0x0000555555581639 in Steinberg::Vst::EditorHost::RunLoop::handleEvents (
    this=0x5555557f3960 <Steinberg::Vst::EditorHost::RunLoop::instance()::gInstance>)
    at /home/paul/VS/vst3sdk/public.sdk/samples/vst-hosting/editorhost/source/platform/linux/runloop.cpp:133

[baconpaul]

OK I just spent a bit of time debugging this. There's two basic problems

1: The runloop handling in VST3 on Linux is all done by having the processor vs editor split which we didn't do

1. Trying to fake it by whacking a runloop in place gives you a screen that doesn't repaint

The stack above happens because the runloop isn't initialized. You can fix that by making a runloop object and passing it down to the frame constructor in SurgeGUIEditor like they do in vst3editor.cpp but nothing actually runs that loop them.

So the trick @falkTX played with the aeff editor forcing loop for VST2 needs to happen here, but it needs to happen in conjunction with us embracing the VST3 editor/processor split as described in #164

[baconpaul]

Basically: @falkTX do you know how to hang a runloop in which repaints on vst3 like you did on vst2? If you can point at an example of that it would be lovely.

[falkTX]

I know almost nothing about vst3, cant help much here sorry.
But note that there almost no hosts or plugins that support vst3 on linux right now, so I do not think it is worth bothering.
The hosts that do support vst3 also support vst2, so surge will still work there.
I do not think we need to prioritize this.

At some point later, after initial linux release, I will do a DPF build (like vst2, vst3, etc but for DPF) which allows us to have fully-working standalone (via JACK), LV2 and legal vst2 (no more steinberg vstsdk).
LV2 and VST2 are pretty much the only formats that matter right now on Linux.

[baconpaul]

Ok thanks !

Look forward to the dpf build.

[asimilon]

Perhaps relevant : https://forum.juce.com/t/vst3-support-on-linux/31872/2

[baconpaul]

Thank you yeah that matches what I found exactly. Glad to know I’m not nuts.

But also the vst3sdk contains a runloop for the editor which configures itself pretty well but only if you have the editor / processor split. Since we don’t have that split, we need to c-n-p that editor runloop across (or somehow grossly double inherit or something).

[baconpaul]

Connect with #747

[baconpaul]

OK so here's some details

1. There's a platform/x11/FrameCOnfig which contains a shared_ptr IRunLoop
2. x11platform has a RunLoop::Impl which contains a static ::init method
3. That static ::init method is called by x11Frame in the constructor if the cfg->runLoop is not null
4. That is specified as an option to CFrame::open and is defaulted to =nullptr
5. In SurgeGUIEditor.cpp we call ->open with the 2 arg (so it is nullptr)
6. In the vst2 impl from falktx we do this

{
        frame = new CFrame (CRect (0, 0, 0, 0), this);
        getFrame ()->setTransparency (true);

        IPlatformFrameConfig* config = nullptr;
        X11::FrameConfig x11config;
        x11config.runLoop = &LinuxRunLoop::instance();
        config = &x11config;

    // printf("%s %p %p\n", __func__, frame, ptr);
        getFrame ()->open (ptr, kDefaultNative, config);

where LinuxRunLoop is an implementation of IRunLoop

so looks like VST3 could work if

1. We made SurgeVST3LinuxExtensions which copied/modified quite a bit of the vst2 runloop
2. Ifdef that ->open() for VST3 && Linux to grab a config and pass it in in SurgeGUIEditor
3. Spin up a thread to call an ::idle we add to our runloop implementation every 1/30th of a second or what not

Bumping this to 1.6.2 since I am tired of not having vst3 all platforms.

[baconpaul]

Nope. Too hard.

[baconpaul]

Cool. Steinberg::Linux::IRunLoop and VSTGUI::X11::IRunLoop

One is from the host side and the other is from the plugin sude

That's what all that stupid vst3editor code is bridging

This stuff is a nightmare, undocumented, and badly designed. I'll checkpoint as close as I got on a branch and then try again some other time.

[baconpaul]

Note to self; The linux vst2 calls idle from the synth idle. I wonder if I should do that rather than spin up a thread in the vst3 equivalent (see where I spin up the thread in the ::start in the vst3 support thing). Just for when I come back to that, saw it in the code and thought of it.

[tank-trax]

as a reminder,

the Surge VST3 loads in Reaper and plays presets
in Bitwig it gives a black screen and can only play the loaded init

[baconpaul]

In bitwig you canauromste parameters and it responds correctly right?

[baconpaul]

Can automate that is. Sorry

[tank-trax]

what I notice is the VST3 does not display values but fiddling with the settings will produce changes in sound

here is what the VST2 looks like

and the VST3

[baconpaul]

Oh that’s super useful

I will bang my head against it one more time before we ship 1.6.2. Maybe I can find it.

[baconpaul]

Right so debugged some. Here's the difference

• both reaper and bigwig are registering a file descriptor listener and 3 timers.
• both reaper and bitwig are running the timers
• but bitwig never calls us back in an event on the file descriptor
• and bitwig unregisters 2 of the 3 timers twice

Hmm.

[jpcima]

The file descriptor is supposed to activate on Xcb events arriving, which in some particular cases poll() failed to notice.

For example under Lv2 idle callback, polling is never working at this call site. But you can get it working in a parallel thread. (which no doubt is somewhat racey, but then it works)

I guess something else may be consuming the events before that VSTGUI is able to poll them.

[baconpaul]

Just hitting this with some notes to self

As @tank-trax points out, in some hosts (Carla it seems) and sometimes in bitwig the menu doesn't work. This is also the case in the JUCE6 AudioPluginHost which is a super useful testbed

cmake -Bbuild -DJUCE_BUILD_EXAMPLES=ON -DJUCE_BUILD_EXTRAS=ON
cmake --build build --config Release --target AudioPluginHost
./build/extras/AudioPluginHost/AudioPluginHost_artefacts/AudioPluginHost

and then you can load a surge vst3. I see

1. menus don't work
2. close and reopen are not reliable and more improtantly
3. close leaves a ghost window around

[baconpaul]

So after a bunch of work with @tank-trax and many others, I'm now confident that the surge VST3 is basically fine at head in the nightly. One or two glitches here but those are separate issues. So I'm gonna close this