Surge: Linux - Heavy CPU while Surge LV2 UI is kept open under Ardour

[xard-dev]

Describe the bug
Heavy CPU usage while Surge LV2 UI is open in Ardour 6. Weirdly when kept open other open plugins don't idle update themselves.

This is not issue on other hosts than Ardour and the VST2 version Surge does not cause this either.

Please let us know your surge version

• Surge Version: 1.7 Nightly 24f7676
• Plugin type LV2
• Bits 64

To Reproduce

1. open Ardour6
2. create new project
3. insert new midi track with Surge LV2 plugin
4. open Surge LV2 plugin UI
5. check CPU usage

Expected behavior
Surge LV2 UI being open on Ardour does not consume overly much resources and behaves as Surge VST2 or same Surge LV2 plugin in other hosts.

Screenshots

Desktop (please complete the following information):

• OS: Ubuntu 20.04 LTS
• Host Ardour
• Version 6

Additional context
I've had a quick talk with Ardour devs presenting the issue and I checked basic redraw functionality and LV2 idle function call and they both seem to be okay:

• LV2 idle is called when Surge is idling
• Redraws only happen when something happens on Surge UI

[baconpaul]

Man I sure do wish ardour would add vst3 support soon! We have an increasing backlog of lv2 issues. Thanks for the report!

[xard-dev]

I heard developers mentioning that vst3 support "should" be coming before Ardour 7 but practically that it might take some time before the feature is implemented, tested and ready for use.

I still find it weird that this problem only happens with Ardour and not on other hosts. The issue might not even be on Surge side.

I probably need some help from LV2 developers or look some other LV2 projects to find clues what might be wrong in here. I suspect it has something to do with the LV2 Idle handling as the code is bit peculiar.

[baconpaul]

Can you run a basic time based profiler and at least see which frames are taking time? On mac I have instruments which is a wrapper on dtrace which makes things super easy. I haven’t run dtrace on linux myself, only Solaris and macOS, but I understand it is well supported.

If not that’s fine too. But just perhaps i could look at a time-stack and offer some thoughts.

[tank-trax]

Debian Buster

In both Ardour and Mixbus had the same behaviour

Tested Surge LV2 in jalv.select using jalv.gtk and jalv.gtk3

• same spikes with jalv.qtk and none with jalv.gtk3

• jalv.qtk uses GTK2 and jalv.gtk3 uses GTK3

• Ardour is built with GTK2

Ardour will not migrate to GTK3 as per this article which has the sub headline

Dev sticks with elderly GTK+2 GUI toolkit because 'we would gain nothing' by upgrading

• tested the LV2 in Qtractor and it had no issues. Qtractor uses QT5

• no issues with the Carla which uses QT5

This leads me to believe that the problem is with Surge LV2 and GTK2

[baconpaul]

Super useful thank you

[xard-dev]

That is interesting find!

I had no idea GTK2 was involved in the LV2 build. It's probably not in the VST2 build as there's not problem.

[baconpaul]

We don’t use gtk anywhere in surge but I wonder if the lv2 idle runs at a different frequency

[baconpaul]

As a result of gtk2 that is

[xard-dev]

well, the idle is called frequently, but not that frequently... maybe I should take a look does the idle loop frequency differ to other hosts though.

[xard-dev]

The execIdle function call rate is about same if not faster in Carla than Ardour so it's probably not that at least.

[baconpaul]

OK!

[baconpaul]

thanks!

[tonilink]

Hi. I have no knowledge about LV2, GTK, etc, but when I try to reproduce the bug whit Ardour 6 in a Linux terminal, these two alerts was shown:

GLib-GObject-WARNING *: invalid unclassed pointer in cast to 'GtkWidget'
Gtk-CRITICAL *: IA__gtk_widget_queue_resize: assertion 'GTK_IS_WIDGET (widget)' failed

Whit the version 1.6.6 discontinuing the VST2 version and LV2 whit this bug, it becomes impossible to me to user Surge. I don't know how, but I want to help find out the problem.

[baconpaul]

1.6.6. vst2 is not going away. just 1.7 will not have a vst2.

in bitwig, reaper, qtractor, carla, and several other linux hosts, daw providers have added linux vst3 support, which works well with surge. Perhaps chime in on this thread https://discourse.ardour.org/t/vst-3-support/88241 to help encourage vst3 in ardour, which seems to be the biggest linux daw not moving to vst3?

[baconpaul]

Oh and if you have a valid vst2 sdk license you can build a 1.7 surge from source also. Just we arent distributing a binary

[xard-dev]

Hi. I have no knowledge about LV2, GTK, etc, but when I try to reproduce the bug whit Ardour 6 in a Linux terminal, these two alerts was shown:

GLib-GObject-WARNING *: invalid unclassed pointer in cast to 'GtkWidget'
Gtk-CRITICAL *: IA__gtk_widget_queue_resize: assertion 'GTK_IS_WIDGET (widget)' failed

Whit the version 1.6.6 discontinuing the VST2 version and LV2 whit this bug, it becomes impossible to me to user Surge. I don't know how, but I want to help find out the problem.

Hmm, this could be something as Ardour6 does something different than other hosts related to the window.

Whenever LV2 surge instance window is closed I get the same error messages:

(ardour-6.0.0:6415): GLib-GObject-WARNING **: invalid unclassed pointer in cast to 'GtkWidget'
(ardour-6.0.0:6415): Gtk-CRITICAL **: IA__gtk_widget_queue_resize: assertion 'GTK_IS_WIDGET (widget)

But also whenever I open a LV2 surge instance Ardour6 remembers the window size but as LV2 surge cannot restore some values currently the zoom value remains at default. This results as window which is not completely filled.

For example Carla does not do this at all and there are no GTK errors either.

[baconpaul]

Yeah this is clearly a gtk host problem and I think only ardour is gtk based

[tonilink]

I tested Surge 1.6.6 and Nightly today in Zrythm, both run perfectly, but Zrythm uses GTK3, so I think the problem is GTK2 and LV2 (that JalvGTK and Ardour uses). And the problem is not only the CPU usage, in my computer when I open Surge LV2 inside Jalv or Ardour the RAM peaks 2GB above the average usage in seconds.

[baconpaul]

OK. We don't use GTK in surge at all (we basically use xcb / xembed and cairo through vstgui with no coupling to a large toolkit) - so is there anything we can do here?

[tank-trax]

This issue occurred well back when LV2 Surge first came out, something I noticed when using Jalv.Select with jalv.gtk as the selector and that it did not occur when using jalv.gtk3

I am not an avid Ardour user and generally when using Mixbus it is for final mixes so it had not caught my attention with the Surge LV2. That being said I think this is a longstanding issue.

I haven't a clue as to what GTK3 does different than GTK2 and why there is less CPU and memory spiking.

This issue has also been presented to Ardour forums by @tonilink

[baconpaul]

That forum seems to indicate the LV2 idle loop could do something special.

Anyone here know anything about LV2 idles?

I dont

[baconpaul]

Anyway in lv2/SurgeLV2Vstgui.cpp it seems we call the UI event handler on each and every idle. We don’t know if there’s been an update of course (since xcb polling doesn’t work) but I wonder if that is too often.

I would try the following

1. Put a print or count in SurgeLV2Vstgui.cpp where it says “eventHandler->onEvent()”. That’s what retriggers the xcb refresh (although that xcb refresh should be minimal)

2. If that’s happening 2000 times a second then throttle it. Run it most 100 times a second or so. std::chrono makes that easy enough plus a little state. Or even 50 times a second.

Unlike the VST3 (where we had the slow repaint because of not collapsing events) it looks like the LV2 authors do, indeed,give us a possibility of a runaway refresh which would match what you see and what’s in that thread.

So I’d start there myself.

[xard-dev]

As far as I know the LV2 idle loop is called at steady rate on Ardour 6 and in Carla as well. The rate might be bit different but I'm not sure if the code is supposed to stay in the LV2 loop.

Also the resize operation problem when the window is closed might be an indication that the GTK gets stuck on widget resize operation and gives up only after the window is closed. Another hint is that all unrelated plugins also stop refreshing draws in background as long as a Surge LV2 window is open. (someone else could also confirm if this is the case for everyone)

[baconpaul]

yeah i can't even make ardour load surge so I'm no help here. But I would try throttling the idle loop.

I dunno. The LV2 doesn't work very well. The folks who wrote it are busy with other stuff. I think 1.7.0 on linux for lv2 and vst2 only users is going to be an upgrade they don't particularly take to unless they can build their own VST2.

[xard-dev]

I can compile latest git lv2 version (and probably vst3 but due to lack of working hosts cannot test currently)

[baconpaul]

Yeah I can compile it too but I’ll be darned if I can figure out how to make ardour5 find it when I add a track.

[xard-dev]

Ardour6 calls idle loop around 950 times within 30 seconds. (sorry for the bit arbitrary timing here)

...and Carla calls it 1509 times during 30 seconds. Which aligns with my empirical estimation it being faster without counting.

[baconpaul]

ok so that's not it then.

[xard-dev]

Someone had used jalv (which is simple lv2 hostby the way) and the version I have in the repository does not have gtk2 version.

When tested with jalv.gtk3 and jalv.qt5 Surge LV2 does not have the cpu usage issue at all.

I managed to compile older version of jalv.gtk myself and yes, the problem can be reproduced with it. It does not print widget errors as Ardour 6 does but the core issue is clearly the same.

This is actually a good thing as now I have a simple host which I compile and debug where things might get out of hand on the host side.

[baconpaul]

How do you build the gtk2 version? Ubuntu 20 apt gave me the 3 version yup

Really seems gtk2 is the issue though. Also do I read correctly that GTK3 was released in 2011? So GTK2 has been the old version for 9 years? Huh - no wonder Ubuntu doesn’t install it by default!

[baconpaul]

Also when you run jalv.gtk in a profiler does it give you a hotspot in the surge code? If so that would be useful to know

[xard-dev]

Okay, I'm really inexperienced when it comes to debugging and profiling cpp / c applications but I managed produce following chart with help of valgrind and kcachegrind.

Actually this is only the engine part.

[xard-dev]

Here's the UI side CPU usage chart:

[xard-dev]

And finally both combined in in area chart:

[tank-trax]

How does one run jalv.gtk in a profiler?

[xard-dev]

How does one run jalv.gtk in a profiler?

I just ran the jalv.gtk under valgrind:
valgrind --tool=callgrind ./build/jalv.gtk https://surge-synthesizer.github.io/lv2/surge

I think I could get additional information by compiling the jalv with debug symbols but I need to look more information related to this.

[baconpaul]

yeah we should really check it with dtrace ... is there a way to build jalv.gtk on ub20?

[xard-dev]

Current jalv does not compile on with system libraries on ubuntu 20.20 due to the system repositories having older lv2. I tested with 1.6.2 which did compile: http://drobilla.net/2019/06/06/jalv-1-6-2.html

Once you run ./waf configure it reports the additional libraries needed which are the usual stuff like libsuil, libsord etc.

Even though the configure lists other frontends only one of them is needed (like gtk2 dev in my case) needed for the compile.

[tank-trax]

Here are some jalv resources:

https://drobilla.net/software/jalv
https://gitlab.com/drobilla/jalv

for dependencies:

https://gitlab.com/users/drobilla/projects

[xard-dev]

The valgrind output is bit confusing but I managed to notice something when doing a test run with both jalv.gtk and jalv.gtk3 opening the surge and doing absolutely nothing but closing the window until certain amounts of idle loop was being called.

When comparing the output It seems that the jalv.gtk is constantly calling gtk_container_resize_children and gtk3 version was only calling gtk_container_check_resize and zero times any resize functions.

Also the gtk2 version did call resize function every time it did a resize check.

[xard-dev]

After doing some more experimentation I found out that if I set the jalv.gtk alignment container resize mode does indeed make a difference. The alignment container is a gtk component which stores the plugin UI and aligns it inside plugin window.

When the alignment resize mode is set to IMMEDIATE before adding the plugin widget to the container the alignment container does not ask the widget to resize itself during initalization.

gtk_container_set_resize_mode((GtkContainer*) alignment, GTK_RESIZE_IMMEDIATE);

After patching jalv.gtk code the CPU usage only starts after window is resized or zoom level is changed so that the window size will be altered.

So the issue seems to be that the GtkWidget instance created by suil_instance_get_widget promises widget resize but the logic goes off the rails when a resize request reaches the widget component.

[xard-dev]

And yes, Surge does indeed report http://lv2plug.in/ns/extensions/ui#resize as feature but I think that this is a two way street: The plugin can ask the container to be resized but the container can also ask the plugin to be resized.

[tank-trax]

OS: Debian Buster
DE: KDE Plasma

recently noticed that when running Surge LV2 using jalv.gtk3 https://surge-synthesizer.github.io/lv2/surge the GUI cannot load, returning this error

(jalv.gtk3:9138): Gdk-ERROR **: 14:05:36.049: The program 'jalv.gtk3' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadWindow (invalid Window parameter)'.
  (Details: serial 189 error_code 3 request_code 3 (core protocol) minor_code 0)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the GDK_SYNCHRONIZE environment
   variable to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)
Trace/breakpoint trap

the following are able to display the GUI

jalv.gtk https://surge-synthesizer.github.io/lv2/surge
jalv.gtkmm https://surge-synthesizer.github.io/lv2/surge
jalv.qt4 https://surge-synthesizer.github.io/lv2/surge
jalv.qt5 https://surge-synthesizer.github.io/lv2/surge

[baconpaul]

So turns out the ttl had been wrong for a while. I just pushed a fix to fix it. Any chance you can try again at head of main?

[tank-trax]

commit fc8256

still can't load Surge LV2 using jalv.gtk3 (same error), still getting big CPU spikes with jalv.gtk and jalv.gtkmm, QT4 is best, QT5 opens but needs to be resized, once resized runs well

[baconpaul]

Thanks. Did you try it with gdk synchronize set out of curiosity?

[tank-trax]

Sorry to ask. How does one run it with the GDK_SYNCHRONIZE environment variable?

[tank-trax]

I ran

valgrind --leak-check=full jalv.gtk3 https://surge-synthesizer.github.io/lv2/surge

and got this

surge.lv2.error.txt

[xard-dev]

Managed to compile new main and update all submodules (hopefully) and for me jalv.gtk3 does work as expected like before and does not crash but I still have the same CPU issue as before with gtk2.

I'd need to double check what was updated from the ttl manifest files... pointing to commit: fc8256b5a3f6a72887fb949554c48c0416ec1994

By the way is it correct to have the commas in the ttl generator without space?

"    lv2:optionalFeature ui:parent,\n"
              "                        ui:resize,\n"
              "                        ui:noUserResize ;\n"

[tank-trax]

Glad to see you got the jalv.gtk3 working, so it must be a local issue. Need to start digging.

[tank-trax]

and it was a local issue and it's fixed

jalv.gtk3 https://surge-synthesizer.github.io/lv2/surge loads the GUI now

[xard-dev]

I finally managed to get debug symbols using GDB working with jalv-gtk and surge. For some reason it was incredibly difficult to get Surge to compile debug flags no matter what cmake variabes I tried to use. Only way I got it working was by changing the value from CmakeLists.txt: set(CMAKE_BUILD_TYPE Release) > set(CMAKE_BUILD_TYPE Debug)

After that Surge finally compiled with debug symbols enabled and after getting glib2 and suil debugsymbols I started getting readable results from GDB about the bindings between jalv and surge.

Here's the difference I noticed when comparing the SuilX11Wrapper:

// SuilX11Wrapper Surge
max_size = { is_set = false,  width = 0,  height = 0 }, 
custom_size = { is_set = true,  width = 904,  height = 542 }, 
base_size = { is_set = false, width = 0, height = 0 }, 
min_size = { is_set = false, width = 0, height = 0 }

Comparing to any other LV2 plugin I loaded I noticed that they ALL had min_size and max_size defined:

// SuilX11Wrapper ADL Plug (one of many tested)
max_size = { is_set = true,  width = 800,  height = 600 }, 
custom_size = { is_set = true, width = 800, height = 600 }, 
base_size = { is_set = false, width = 0, height = 0 }, 
min_size = { is_set = true, width = 800, height = 600 }

By digging through SUIL wrapper code I noticed that during initialization suil_x11_on_map_event expects min_size hint to be defined to work around GTK business.

[xard-dev]

Curiously enough vst3sdk used in Surge also always sets window min_width and min_height:

if (!resizeable)
{
    sizeHints->flags |= PMaxSize;
    sizeHints->min_width = sizeHints->max_width = size.width;
    sizeHints->min_height = sizeHints->max_height = size.height;
}
else
{
    sizeHints->min_width = sizeHints->min_height = 80;
}

[tank-trax]

good work @xard-dev
@baconpaul these findings may be useful for the LV2 not saving the zoom sizes issue

[baconpaul]

Yeah! Let me tag in @jpcima and @falkTX - I have no idea where in the startup path of LV2 these would be set but clearly there's some init time call missing.

[xard-dev]

@tank-trax thanks!

I tried to look where the LV2 window hints are set but could not found clear indication of that in the codebase: I'd like to test adding the information before handing the Surge to SUIL wrapper happens.

[jpcima]

Hi, based on a very quick examination, I notice an anomaly.
See at https://github.com/surge-synthesizer/surge/blob/dfb63d7d7738eaa71e863258d7875b89e0f814b1/src/lv2/SurgeLv2Ui.cpp#L78

The plugin is supposed to write its Window ID at the location pointed by this pointer.
(ie. the child window HWND, or OS equivalent)

It's a similar mistake as I did in another plugin recently.
Most of hosts accept to let it work regardless, but this might be at the root of some subtler issues.

[baconpaul]

Thanks @jpcima - do you know the incantation which would work? Like: What window type does the LV2 expect?

Excellent debugging.

[xard-dev]

Snippet from dragonfly reverb:

if (widget != nullptr)
            *widget = (LV2UI_Widget)fUI.getWindowId();

I tried in GDB that inserting parentWindow in to the widget pointer did not do the trick but the Surge started just fine despite that.

[jpcima]

Not the parent window, but the child one.
From SurgeLv2Ui that would probably be _editor->getFrame()->getPlatformFrame()->getPlatformRepresentation().

[xard-dev]

hmm...

_editor is not available at the SurgeLv2Ui::instantiate context and if I try to access instance _editor following happens:

(gdb) p instance->_editor->getFrame()->getPlatformFrame()->getPlatformRepresentation()
Couldn't find method SurgeLv2Ui::getFrame

[xard-dev]

@jpcima I think I figured it out:

The _editor comes available only after ui instance has been initialized which comes in the very end of the instantiate function so I added following patch to the end (taking same precaution as dragonfly reverb plugin):

std::unique_ptr<SurgeLv2Ui> ui(new SurgeLv2Ui(instance, parentWindow, featureUridMap,
                                                 featureResize, write_function, controller, uiScaleFactor));  
if (widget != nullptr) {
     *widget = ui->_editor->getFrame()->getPlatformFrame()->getPlatformRepresentation();
}
return (LV2UI_Handle)ui.release();

Guess what?

It works!

[xard-dev]

@jpcima I need to ask that what was the source for this widget maneuver information?

I read through the LV2 documentation and it mentions no such thing.

[jpcima]

@xard-dev source is this API reference
https://lv2plug.in/doc/html/group__ui.html#ad0fc716947519b344414ca5d4b35ecf4

[baconpaul]

Wahey! Excellent!

@xard-dev happy to merge a PR with that diff if you want to fire it in. If you do, please add yourself to the AUTHORS file in the request (or are you there already? Sorry if so!)

[xard-dev]

Okay, I'll have to check how to do that but I'll add the lv2 documentation comment related to the widget as I'm not capable of comprehending how it could mean this...

[baconpaul]

The git howto in the doc directory explains it all! Or alternately clean up your changes and type 'git diff' and attach the output here and we can walk it through for you.

[xard-dev]

I noticed that doing local fork and then pushing to pull request is bit too much currently so I'll take the diff file route:
SurgeLv2Ui_cpu_patch.zip

It's done against latest main release... now that I got it pulled properly.
(For some reason I have also lot of trouble updating the git submodules and they always seem to go to modified state instead of clean)

[baconpaul]

OK all merged into main. Thanks for the careful debugging here!

[tank-trax]

I tested in jalv.gtk and no more lagging
tested in Ardour and the LV2 and VST2 have the same level of CPU spiking when the GUI is moved around, which is way less spiking then occurred prior to this fix

[xard-dev]

@baconpaul Thanks!

[tank-trax]

@xard-dev thanks to you too