Nemo: Crashes in favorite_check_callback

[joequant]

This may be the same issue as #2575. It appears that something is getting double freed

 * Nemo version (nemo --version)

4.8.0
 * Is issue with desktop or windowed nemo?

desktop
 * Distribution - (Mint 17.2, Arch, Fedora 25, etc...)

Mageia caudron

 * Graphics hardware *and* driver used

Intel laptop

 * 32 or 64 bit

64
 ```

**Issue**

Tracking down crash in favorite_check_callback.  It appears that the ref count is getting broken somewhere....


**Steps to reproduce**

nemo-desktop


**Expected behaviour**

Crash

Here are some valgrind stack traces

==12077== Invalid read of size 8
==12077== at 0x4C60C1: favorite_check_callback (nemo-directory-async.c:3453)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Address 0x70bfde0 is 0 bytes inside a block of size 24 free'd
==12077== at 0x48378E9: free (vg_replace_malloc.c:538)
==12077== by 0x55CF678: g_free (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Block was alloc'd at
==12077== at 0x4836751: malloc (vg_replace_malloc.c:307)
==12077== by 0x55CF568: g_malloc (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:3543)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:4601)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:4646)
==12077== by 0x4C1AB4: nemo_directory_async_state_changed (nemo-directory-async.c:4628)
==12077== by 0x4C3ACE: UnknownInlinedFun (nemo-directory-async.c:3672)
==12077== by 0x4C3ACE: link_info_nemo_link_read_callback (nemo-directory-async.c:3722)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7F7A: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53A6EF1: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53C427D: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7408: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077==
==12077== Invalid read of size 8
==12077== at 0x4C61BE: UnknownInlinedFun (nemo-directory-async.c:3438)
==12077== by 0x4C61BE: favorite_check_callback (nemo-directory-async.c:3495)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Address 0x70bfde8 is 8 bytes inside a block of size 24 free'd
==12077== at 0x48378E9: free (vg_replace_malloc.c:538)
==12077== by 0x55CF678: g_free (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Block was alloc'd at
==12077== at 0x4836751: malloc (vg_replace_malloc.c:307)
==12077== by 0x55CF568: g_malloc (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:3543)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:4601)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:4646)
==12077== by 0x4C1AB4: nemo_directory_async_state_changed (nemo-directory-async.c:4628)
==12077== by 0x4C3ACE: UnknownInlinedFun (nemo-directory-async.c:3672)
==12077== by 0x4C3ACE: link_info_nemo_link_read_callback (nemo-directory-async.c:3722)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7F7A: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53A6EF1: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53C427D: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7408: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)

==12077== Invalid read of size 8
==12077== at 0x55559D5: g_type_check_instance_is_fundamentally_a (in /usr/lib64/libgobject-2.0.so.0.6600.3)
==12077== by 0x553AA44: g_object_unref (in /usr/lib64/libgobject-2.0.so.0.6600.3)
==12077== by 0x4C61C7: UnknownInlinedFun (nemo-directory-async.c:3438)
==12077== by 0x4C61C7: favorite_check_callback (nemo-directory-async.c:3495)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Address 0x70bfe70 is 48 bytes inside a block of size 88 free'd
==12077== at 0x48378E9: free (vg_replace_malloc.c:538)
==12077== by 0x55CF678: g_free (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55E743F: g_slice_free1 (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55547A3: g_type_free_instance (in /usr/lib64/libgobject-2.0.so.0.6600.3)
==12077== by 0x4C61C7: UnknownInlinedFun (nemo-directory-async.c:3438)
==12077== by 0x4C61C7: favorite_check_callback (nemo-directory-async.c:3495)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Block was alloc'd at
==12077== at 0x4836751: malloc (vg_replace_malloc.c:307)
==12077== by 0x55CF568: g_malloc (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55E6C9D: g_slice_alloc (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55E7319: g_slice_alloc0 (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55544A9: g_type_create_instance (in /usr/lib64/libgobject-2.0.so.0.6600.3)
==12077== by 0x553B0F4: ??? (in /usr/lib64/libgobject-2.0.so.0.6600.3)
==12077== by 0x553C48C: g_object_new_with_properties (in /usr/lib64/libgobject-2.0.so.0.6600.3)
==12077== by 0x553CF40: g_object_new (in /usr/lib64/libgobject-2.0.so.0.6600.3)
==12077== by 0x4C1ABF: UnknownInlinedFun (nemo-directory-async.c:3545)
==12077== by 0x4C1ABF: UnknownInlinedFun (nemo-directory-async.c:4601)
==12077== by 0x4C1ABF: UnknownInlinedFun (nemo-directory-async.c:4646)
==12077== by 0x4C1ABF: nemo_directory_async_state_changed (nemo-directory-async.c:4628)
==12077== by 0x4C3ACE: UnknownInlinedFun (nemo-directory-async.c:3672)
==12077== by 0x4C3ACE: link_info_nemo_link_read_callback (nemo-directory-async.c:3722)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7F7A: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53A6EF1: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53C427D: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7408: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077==

(nemo-desktop:12077): GLib-GObject-CRITICAL **: 16:45:26.961: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
==12077== Invalid free() / delete / delete[] / realloc()
==12077== at 0x48378E9: free (vg_replace_malloc.c:538)
==12077== by 0x55CF678: g_free (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Address 0x70bfde0 is 0 bytes inside a block of size 24 free'd
==12077== at 0x48378E9: free (vg_replace_malloc.c:538)
==12077== by 0x55CF678: g_free (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077== Block was alloc'd at
==12077== at 0x4836751: malloc (vg_replace_malloc.c:307)
==12077== by 0x55CF568: g_malloc (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:3543)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:4601)
==12077== by 0x4C1AB4: UnknownInlinedFun (nemo-directory-async.c:4646)
==12077== by 0x4C1AB4: nemo_directory_async_state_changed (nemo-directory-async.c:4628)
==12077== by 0x4C3ACE: UnknownInlinedFun (nemo-directory-async.c:3672)
==12077== by 0x4C3ACE: link_info_nemo_link_read_callback (nemo-directory-async.c:3722)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7F7A: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53A6EF1: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53C427D: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F73C0: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x53F7408: ??? (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077==

ERROR:../libnemo-private/nemo-directory-async.c:3462:favorite_check_callback: assertion failed: (NEMO_IS_FILE (favorite_check_file))
Bail out! ERROR:../libnemo-private/nemo-directory-async.c:3462:favorite_check_callback: assertion failed: (NEMO_IS_FILE (favorite_check_file))
==12077==
==12077== Process terminating with default action of signal 6 (SIGABRT): dumping core
==12077== at 0x5BB7450: raise (in /usr/lib64/libc-2.32.so)
==12077== by 0x5BA2525: abort (in /usr/lib64/libc-2.32.so)
==12077== by 0x5595DCB: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55F0E5A: g_assertion_message_expr (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x4C6127: favorite_check_callback (nemo-directory-async.c:3462)
==12077== by 0x55C55D6: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C975D: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9AB7: ??? (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x55C9B5E: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.6600.3)
==12077== by 0x5424194: g_application_run (in /usr/lib64/libgio-2.0.so.0.6600.3)
==12077== by 0x42F23E: main (nemo-desktop-main.c:101)
==12077==
```

Other information

[joequant]

It looks like some sort of race condition between two callbacks.

[mtwebster]

Can you reproduce this reliably?

I've tracked down what I'm about 95% certain is the cause but I'd like to confirm it before I push the fix.

Thanks

[joequant]

Yes. I can reproduce it. Give me a patch and I'll tell you how it goes.

I think the difficulty in reproducing has to do with the fact that there needs to be a particular configuration of directories and favorites to cause it to fail, but my home directory has that set of directories, so nemo crashes every time I run it.

Also, I'm packaging cinnamon 4.8 for Mageia so this is a blocker for me.

[Cyanoxygen]

Hi, I ran into this similar problem while packaging the cinnamon desktop environment.

Platforms tested

I have two platforms tested:

• amd64, GCC 10.2.1
• arm64 , GCC 10.2.1

Issue

The issue is mainly on arm64 side:

arm64

Environment

Raspberry Pi 4, running arm64 build of AOSC OS.

nemo can't launch. It flashes the main UI rapidly, with a "loading" in it, then it quitted immediately, thrown a SIGSEGV.
I tried to reset the favorites entry in gsettings, but it didn't help either.

I tried gdbing but definitely, I can't get a clear result because all the symbols are stripped and codes are optimized out. Then I built nemo with -O0 -g, performed a live gdb debugging.

According to gdb's report, the crash point is around favorite_callback_check. Sometimes it crashed a little bit furtherer, sometimes it crashed earlier.

At this time the memory pointed by directory is not accessible.
We discussed this possible bug in our chat group, one of our members helped and guided me on some debugging work.
We finally set a breakpoint at libnemo-private/nemo-directory-async.c:3549, where the function call occurred, and discovered that this breakpoint hit twice before the crash, in order to 'catch' the values (specifically, the state) passed to the function.

The first time it hit the breakpoint, the process is able to continue, but after hitting this twice, the program crashes.

I'm not a programmer, but I will try my best to help you debug this problem.

[joequant]

Yup. Same problem. Do you have a patch we can use to fix?

[joequant]

@mtwebster Hi. Just checking about the status of the patch. I'm pretty interested in what the cause of the problem is. Race condition?

[mtwebster]

Ok I made some packages with some changes that may address this issue - the changes I made here result in safer behavior than what I had before, so it's worth trying, if only to gather additional data:

https://www.dropbox.com/sh/vec4w2jhgfa221r/AADnUuKCa25sV60STU6srUS3a?dl=0

Download the packages to a folder and use dpkg -i, don't try to open via the browser.

Try to install nemo, nemo-data, libnemo-extension1 and gir1.2-nemo-3.0 first. Only install the -dev and -dbg packages if the installer complains about them.

Here's the branch this is built off of for reference:
https://github.com/mtwebster/nemo/tree/favorites-async-job

Thanks

[joequant]

Seems to fix the issue at my end....

[joequant]

Looks good!!!! Thanks!!!!

[Cyanoxygen]

Really appreciate it, then I will build one and have a test with it now! :D

[Cyanoxygen]

Testing

Raspberry Pi 4, arm64 AOSC OS.
DE: Openbox (the upper image), Cinnamon 4.8.0 (checked out from tags, the lower image).

It now runs smoothly! And it won't crash anymore, even after adding and modifying favorites.

[mtwebster]

I merged this patch:
https://github.com/linuxmint/nemo/commit/933545507fa3ab15a90abe647f517000d2011022

Closing this - if any more issues pop up we can re-open or do a new report.

Thanks for testing!