Lmms: LMMS hangs when certain VST effects are reloaded in a saved project

[musikBear]

(0.92 xp 32 bit)
I looked at one of the vst-instruments (Kairatune.dll) that lmms does work with. The vst loads correct in an empty vst-container, but after project is saved, it is imposible to reopen. So is any saved instrument made (.xpf) with the vst.
I believe the reason is the chunk in the vst definition
An analyse shows enourmous size of this entry:

Lines 1
Non-empty lines 1
Characters 1728819
Characters without blanks 1728819
Commented char count 0
Commented char count percent 0

the result is the same for both instrument .xpf, and for .mmp -eg same numbers
What could be the reason for lmms to generate a chunk of this size?
If that could be understood, lmms could perhaps load this and other vsts. that does not work now
(btw - teh same chunk in a Synth1 project is 3500 char in size.
(zipped files :http://www.sendspace.com/file/8mmpjj

[softrabbit]

What exactly happens? You mentioned the word "crash", does LMMS crash when loading the project or preset?

[musikBear]

yes, loading a project, or a saved preset makes lmms freeze and "stop responding" (error popup and the whole win blah). A manual shutdown through taskManager is only way out.
-and ofcause that happens. The vst tryes to make lmms read 1.72+ mio ! chars, as information for how some 20 controllers should be posistioned. (? absurd) It is most likely not 'our' bug, and diizy says that the chunk is what comes 'out' of the vst-blackbox, nothing lmms can control. It would be kind of nice to know what the context is.. I have send a message to the creator of the vst. I also looked for repeated sequenses in the chuck, and there seams to be a pattern, but -not- exact. the pattern would be exact, if lmms did some recursive erronious rerad/write, so not that either. If you have a flash of genious, do share

[softrabbit]

The chunk seems like a full dump of all patches in Kairatune. There are strings visible after base64 decoding it.

After looking at the code I have a feeling this might be a limitation in Qt's XML handling, an upper limit on the size of attributes. OTOH, it could be the base64 encoding that fails as well, but that's something used for stuff like e-mail attachments where large files aren't rare.

[Umcaruje]

@musikBear can you confirm this issue still exists? There have been some changes in the way vst data is handled in #1480.

[musikBear]

@Umcaruje i dont have a binary Master. Im still on 1.1.3 off. release, so sorry no.

[musikBear]

@Umcaruje unfortunately Master still hangs when included chunk-heavy VST-effect is attempted reloaded in a saved project
Issue not solved
(tested with BlueCat effect-plugins on win 32x

[tresf]

I can't locate the exact conversation, but we need to fix the way VST presets are loaded in a few circumstances. It is likely this is the cause for a few other bug reports as well.

Possibly related #1183 #1389 #260 #328 forums/#1609

[musikBear]

a question for the dev that implemented the array of automateable VST controllers:
Is there is a way to discover, if a controller on the VST is saved as a automateable lmms-controller in that array?

[PhysSong]

3776 have fixed various VST-related hangs/deadlocks and it will be included in rc4, which will be likely to be released very soon. @musikBear Can you test it with rc4 once it is released?

[musikBear]

Can you test it with rc4 once it is released?

@PhysSong Yes, i can test win 32, just make a '-@' with my name, if/ when you have a binary

[tresf]

-@musikBear here you are https://github.com/tresf/lmms/releases/download/v1.2.0-rc4/lmms-1.2.0-rc3.49-win32.exe

[DomClark]

I've tested this with the latest stable-1.2; Kairatune still fails to load, but LMMS doesn't hang anymore. I expect this is due to this passing true to waitForMessage in VstPlugin::loadChunk, so LMMS notices that RemoteVstPlugin has crashed and stops waiting on a reply. I think I have a fix for the underlying issue, however, so I will put in a PR tomorrow after some more testing if it doesn't seem to break any other plugins.

[musikBear]

@tresf -Thanks i will test those on the "Not-working" List, and the BlueCat-FX-collection

[musikBear]

@tresf Install fails:

First fail is with the QT5.dll
-Btw - is this RC4 ? - its named RC3 (4.9)

[musikBear]

Tried once more after a reboot, and now it installs
Should ofcause have rebooted before install 🤡

[DomClark]

The issue is in RemoteVstPlugin::loadChunkFromFile.

Currently, this tries to allocate memory for the chunk by calling pluginDispatch( 23, 0, 0, &chunk ) (23 is the opcode for getting a chunk from the VST plugin). If this memory is not large enough, the method allocates memory itself. The file is then read into this memory, and it is sent to the plugin via pluginDispatch( 24, 0, _len, chunk ) (24 being the "set chunk" opcode). The issue is a use-after-free; at some point during chunk loading, Kairatune frees the memory it used to return the chunk in the first pluginDispatch call, and promptly segfaults. This can be fixed by always allocating the memory manually in loadChunkFromFile, that is, do away with the first pluginDispatch call and always run the code in the if block.

However, this code was itself introduced in this commit to fix some crashes when setting VST chunks. There's no indication as to which plugins were problematic, but I have yet to find one myself. It's also not clear exactly how the fix worked - did the plugins actually need a call to get a chunk before setting one, or did they just need memory that wasn't freed when the set-chunk call returned? I'm slightly hesitant to revert this as it may well break some other plugins out there, but something needs changing to fix Kairatune. Possible approaches are:

1. Always manually allocate memory and see if any crashes are reported.
2. Always manually allocate memory, but keep it alive throughout the plugin's lifetime in case the plugin needs this.
3. Always manually allocate memory, but keep the "get chunk" call in there in case the plugin needs that.
4. Both (2) and (3).
5. Detect if the plugin is Kairatune (by testing the plugin's unique ID) and treat it separately.

I'd argue against (5) because we could quickly end up with a lot of special cases for a lot of plugins, which would complicate the code somewhat and also possibly break things when plugins get updated.
The issue with (2), (3) and (4) is that it's hard to tell when/whether they're necessary, or indeed which one, and we could end up with bits of useless code hanging around that we have no idea whether they're safe to remove or not. The issue with (1) is that, as previously stated, this could break plugins that have up until now been working. However, on receiving bug reports we could determine the actual cause of the loading errors and then make the necessary fixes.

My inclination is to go with (1) in a beta version, does anyone else have any ideas?

[tresf]

Btw - is this RC4 ? - its named RC3 (4.9)

It's RC3.49, exactly as named. This is RC3 + 49 additional commits. RC4 has not been released.

[tresf]

My inclination is to go with (1) in a beta version, does anyone else have any ideas?

My vote is we do that and monitor the impact. We release emergency fixes for stable releases on occasion and if this is a breaking change, we can handle it that way and simply revert the commit. Furthermore, if this breaks plugins, they will be reported and then we'll (you'll) have data to work with instead of a rather obscure historical commit to go from.

[musikBear]

I am happy to tell you all that RC3.49 loads and reloads BlueCat effects, and analytic tools.

• The presets in blue-cat can be saved
• Individual dials can be saved
• Dials can be automated over the lmms-dial-array

Tested with both BC-Flanger, and BC-FreqAnalyst
This has never been possible before!
I will continue testing other effects, that has been buggy, but alone being able to use the BC-suit, is a big reason for celebrating
GREAT job! 🍾

[PhysSong]

Oops...