Lmms: Project loading-time in RC2.8 significantly longer than 1.1.3

[musikBear]

OS : winXP sp3
LMMS 1.1.90-RC2.8
2 gb ram

I have a project i use as sketchbook. It is 35 mins long.
This project earlier loaded in ~ 50 secd
In RC2.8 this takes ~4.5mins
It looks like lmms uses progressively longer time for longer files
I have made a file over SkiesiOnions that shows this
The file is expanded to 11 mins
Compare the loading of the default SkiesiOnions and the the 11 mins 'version'
On my system the loading halts around 50% and from there it takes ~3 mins to load the project.
This is different from earlier versions of lmms, and i am sure RC2.2 loaded files faster
Is this also happening on better pc's?
A proper benchmark for loading one specific file would perhaps be a good thing to establish, so this feature could be less 'ad hoc' and instead precisely determined on one specified system

File: https://lmms.io/lsp/?action=show&file=10232

[michaelgregorius]

I can confirm that loading got significantly slower between 1.1.3 and the current master (c8e987348defe985253e6ae007cb015abd18f6f1). Here are some manual measurements that have been acquired with a stop watch while loading https://lmms.io/lsp/?action=show&file=10232:
1.1.3: ~4.5s
master: ~9.0s

Some profiling with valgrind shows that most of the delta is spent in MemoryPool::getChunks(int). This method in turn is mainly called by Note::createDetuning().

For a visual overview here is the call graph for 1.1.3:

And here is the call graph for master:

The problem was also described in #2029.

[michaelgregorius]

Commenting out the macro MM_OPERATORS in DetuningHelper also does not help. Loading the file still takes ~9s in this case. So the cost is in fact caused by the sheer number of DetuningHelper instances that are created.

[musikBear]

There is a connection to the new theme
If i choose the Classic theme, i can load the file above in 1.05 min (stop-watch)
@michaelgregorius Can you measure difference in loading of the file above with Classic versus Default theme, and out of curiosity, what are your loading-time(s)?

[michaelgregorius]

When I switch master to the classic theme it still takes ~9s on my machine. The profiler results also do not indicate that there is a problem with the theme. The problem is caused by the DetuningHelpers which have been introduced somewhere between 1.1.3 and master.

[PhysSong]

I think this is because of the our inefficient memory manager. MemoryPool::getChunks() uses very bad algorithm. It really needs improvements.

[zonkmachine]

With the new memory manager the loading speed is something like 2x faster for 'Onion' and 'Krem Kaakkuja'.

[lukas-w]

👍 I think we should still keep this open until the DetuningHelper problem is resolved.

[michaelgregorius]

Opening the project https://lmms.io/lsp/?action=show&file=10232 now takes 7 seconds (stop watch) using a release with debug info build of commit 68c85c8e0579827994759573698c5b27f9c71832.

A fresh valgrind run indicates that the problem with the DetuningHelper seems to be gone. The opening use case is now dominated by calls to Song::updateLength() which in turn calls Track::length() very often:

I assume that too many small updates take place while the file is loading. It would likely suffice to only call Song::updateLength() once at the end when the whole song is loaded.

Here's the full callgrind output:
callgrind-TrackLength.zip

[michaelgregorius]

For some further motivation I have put a return statement at the very top of Song::updateLength to see how this affects the the time it takes to load the test file. It only takes 2 seconds in that case and the GUI state at the end of loading does not even look wrong (although this is definitively not a final fix).

Here's the call graph of that build:

It looks rather "noisy" as is expected for code with no big bottleneck.

Full callgrind output:
callgrind-ImmediateReturn.zip

[musikBear]

I would like to add an observation about windows states.
If i save a very large 'project' (~40 mins -its a scratch-book with ideas) with song-editor open, then i can forget about opening that project again. I will have to convert to mmp, and change state 1 to 0 for 'open', then i can re-open the project. I have had the 'open-state' wait for 60 mins, but it does not open.
There is a peculiarity, because the flag 0 for open, does not mean that the song-editor is closed when it is being populated. The result of the 0-flag, will close song-editor after all the tracks has been loaded, but during loading, song-editor is visible.
I have made it a habit to close all open windows, before i final-save any project, in order to avoid this glitch, -is ofcause hw related, but very real,( witch was why i protested against forced opening of windows)
Most likely noone with more ram than 3 gb will ever have this issue, at all

[tresf]

Most likely noone with more ram than 3 gb will ever have this issue, at all

Can we please stop speculating on specs and performance please? <3

[PhysSong]

It seems fine to do nothing in Song::updateLength while loading projects and calling that once the loading finishes. Any opinions?

[michaelgregorius]

@PhysSong I checked some more and found that the scrollbar of the Song Editor is not updated when I put a return at the top of Song::updateLength. This also matches the observation that the class SongEditor connects its updateScrollBar slot to the lengthChanged signal of the class Song. That signal is emitted at the end of Song::updateLength.

It also has to be made sure that all interactive cases where Song::updateLength is called still work. So far I have found the following methods which call Song::updateLength:

• TrackContentObject::movePosition
• TrackContentObject::changeLength
• Track::removeTCO
• ProjectRenderer::run

I assume that some of them are also called when the user makes some interactive changes to some track content objects.