Rawtherapee: Rotating multiple images simultaneously leads to segfault

[Thanatomanic]

To reproduce: select around 10 to 15 images and use the dedicated buttons in the file browser to rotate them 90 degrees for a few times. This always leads to a segmentation fault, either immediately, or after a few rotations.
It does not seem to happen when you select fewer than 10 images.

Using: latest dev build on Windows 10.

[Thanatomanic]

From gdb:

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00000000005603f9 in BackBuffer::copyRGBCharData(unsigned char const*, int, int, int, int, int, int, int) ()

[sguyader]

No crash here with version 5.4-923-g07dafc67a on Linux. Compiling right from current code to see.

Edit: cannot reproduce on Linux with current version: 5.4-940-gf6039a35b

[Floessie]

It's obivous that there's a intermediate state where the backbuffer is corrupted. This doesn't lead to a crash here on Linux, though.

[Thanatomanic]

@Floessie I have seen distorted thumbnails appearing too, always as a precursor to a crash

[Thanatomanic]

Here's a more complete backtrace using a debug build:
log.txt

[Hombre57]

@Floessie I've spent few hours trying to understand why it crash without success. There's a mutex protecting the access to the image data, so it shouldn't crash.

However the whole threadutils.h file with utility class file that I've created, to circumvent different behavior between OS and add tracing tools (done as a rookie programmer...), is now obsolete because Glib::Threads::Mutex and Glib::Threads::RecMutex is deprecated.

I'm not sure it's the culprit, but someone should open an issue to review that file.

[Hombre57]

I finally managed to understand what's going on. The GUI ask to render the thumbnails with its new orientation while it's still in the processing queue and only the last preview is available, with old orientation, leading to a segfault by buffer overrun.

While trying to fix it, I already converted the threadutils utility class to c++11 std::mutex class, no deadlock so far.

I'll try to fix it in the coming days. Hopefully it should solve issue #5310.

[Floessie]

@Hombre57 So, which was the unprotected variable?

[Hombre57]

@Floessie It's not about an unprotected variable, but an obsolete data that should have been deleted. In fact, the fix is a two liner :slightly_smiling_face: However I've added some more protection to prew and preh in filebrowserentry.cc, and updateBackBuffer, with MYREADERLOCK(l, lockRW);

Not sure it was necessary, but it shouldn't harm and make it more robust :question:

The twoliner was this and this.

Now when you rotate thumbnails, there's no preview and no garbage displayed while it's in the processing queue.

@Beep6581 I'd suggest to add this to 5.7.

[Hombre57]

Forgot to mention : committed in branch rotationusedtocrash

[Hombre57]

And for the record...

Coarse Rotate flow.txt

[Floessie]

Coarse Rotate flow.txt

Phew!

[Beep6581]

@Hombre57 @Floessie @heckflosse ok, but let's merge today then!

[Hombre57]

@Beep6581 rotationusedtocrash merged to dev.