Floccus: Sync failing under 4.4.1, 4.4.2, 4.4.3 now issuing Cannot map parentId errors under v4.4.6

[wohali]

Describe the bug

1. Starting sync on 4.4.4 from toolbar icon click renders a popup window with an endless spinning circle for 10-15s. Eventually, the UI draws itself. (This was never an issue before).
2. Cannot map parentid errors.

To Reproduce
Steps to reproduce the behavior:

1. Start a sync after upgrading Floccus to 4.4.4 and Bookmarks to 3.4.6. [edit: Still happening with 4.4.6 and 3.4.9.]

Expected behavior
Sync should work, which it was doing prior to 4.4 series.

Screenshots

Desktop (please complete the following information):
Windows, Firefox 82.0.2, Floccus 4.4.4, Nextcloud folders,

Server (please complete the following information):
Debian Buster, Nextcloud 19.0.4, Bookmarks app 3.4.6

Note: As your debug log usually contains large parts of your bookmark data, instead of attaching it publicly to the issue here, I recommend uploading it privately at the following URL https://klehranlage.net/index.php/s/7nn82NSBsfQKdsJ

• [X] Debug log provided: floccus-4.4.4-2020-10-30-33.log

Additional context

Prior to 4.4.4 update I was getting root folder errors. Those appear to have stopped.

[dsiminiuk]

4.4.4 installed today over top of 4.4.3 on 8 browsers all sharing one NC bookmarks account using the same path.
This update was without error.

[marcelklehr]

@wohali :disappointed: Have you tried the old "Sync from scratch"?

[wohali]

Yes, several times @marcelklehr . No joy.

I've got one machine working via both delete-all-bookmarks-in-Nexcloud-then-sync-from-scratch. But the next machine, doing sync-from-scratch, results in the Cannot map parentId error.

[wohali]

@marcelklehr Now on the one-working laptop I'm getting #718 .

This is so frustrating, this plugin has been broken almost right since I first installed it. :sob:

[wohali]

I've uploaded the log from the first-working machine (now failing with #718), see 716-floccus-log-2.log

[dsiminiuk]

Export/backup bookmarks on your working machine. Safe keeping.
Disable sync on the working machine.
Delete all bookmarks on the server.
UPLOAD to the server from the working machine to create a fresh list on the server.
Download from the server on the other machines.
This is what fixed a corrupted tree in my situation.

[wohali]

@dsiminiuk Can't do that because I have been out of sync so long, I have disparate bookmarks on all the machines now. I'd lose valuable data that way. Unless I do manual merging of some sort.

Did a delete all on server, then push up via floccus via one, but it still broke on just that one machine as mentioned in this ticket. Other machines haven't been able to upgrade since 4.4.0.

[dsiminiuk]

@wohali All I can suggest in that case is to export the bookmarks from all your machines and import them into one, then use a de-duplicator plugin to get the unique set. Export/backup that set and start from there.

[wohali]

@dsiminiuk Daniel, thank you for your input, but I'd rather that the author of this plugin and application review and consider what can be done to fix the problem. I'm not interested in a workaround that may very well just break again.

[marcelklehr]

@wohali I'm unable to replicate this, sadly, but I can understand your frustration. Something you could try is run occ maintenance:repair on your nextcloud, to clean up the nextcloud db in a non-invasive way.

[wohali]

@marcelklehr Bookmarks app has the issue where every run of occ maintenance:repair prints out:

     - Removed 210 orphaned bookmarks entries

as if it cannot fix them. Current version of Bookmarks app is 3.4.6.

[tessus]

@marcelklehr I also see this now. Cannot map parentId:1336. Remember, I did the repair step and the sync from scratch. This worked. But on a subsequent sync, this error happend.

[marcelklehr]

as if it cannot fix them

Aha, I'm on to this one atm. Meanwhile v4.4.5 is out, which is unlikely to fix this issue, but one can dream...

[wohali]

@marcelklehr OK, the one machine that started working under 4.4 (from the delete-all-and-upload) is back to working with 4.4.5, but all of the others are still failed.

I'll wait for your bookmarks app update and then do an occ maintenance:repair to see if that helps restore the other machines.

[tessus]

I've upgraded to Bookmarks 3.4.7 and this is what happens:

$ occ maintenance:repair
 - Remove orphaned bookmark tree items
     - Removed 124 orphaned bookmarks entries
     - Removed 2317 orphaned folders entries
     - Removed 70 orphaned children entries
     - Removed 38 orphaned bookmark folders
     - Removed 32 orphaned bookmarks

$ occ maintenance:repair 
- Remove orphaned bookmark tree items
     - Removed 86 orphaned bookmarks entries
     - Removed 2257 orphaned folders entries
     - Removed 1934 orphaned children entries
     - Removed 69 orphaned bookmark folders
     - Removed 999 orphaned bookmarks

So every time I run occ maintenance:repair, something else is shown. Even if I run the command in a loop and no sync attempts are made from any clients.
IMO the first run should clean everything up, and the second run should show 0 for all entries. Or am I missing something?

[tessus]

Well, after running the repair command 3 times, all my bookmarks are gone from the server and I get the message "No bookmarks here".
Luckily I do have a backup, but this is messed up. I think nobody else but me uses the bookmark app, so I hope no data was lost. However, this has me really concerned.

Please tell me which tables are created by the bookmarks app. Since all bookmarks are now gone on my server, I will remove all tables manually and reinstall the bookmarks app. Maybe this fixes the problem.

Update: Unfortunately I lost bookmarks, but maybe I can reconstruct some of them from a db backup that is a week old.

[TheFrenchGhosty]

I had this issue since I moved to 4.4.X (or Nextcloud 20, I don't actually know), and well, but I think I fixed it.

This is how:

• Export your bookmarks from Nextcloud Bookmarks settings (from the web UI)
• Disable floccus on the device you're currently using
• Import your bookmarks from your Nextcloud export to your browser
• Run "Bookmarks Dupes" (or any other addons you want) to remove the duplicates (I had some bookmarks only on the Nextcloud export, meaning that I would have lost them if I skipped that)
• Export your bookmarks from your browser (this is your backup)
• Go into Nextcloud Bookmarks settings (from the web UI) and click "Delete all bookmarks"
• Run occ maintenance:repair on the server
• Enable floccus on the device you're currently using
• Go into the settings - Dangerous actions and click "Trigger Sync from Scratch"
• Click sync now in the addon pop-up
• Wait for it to be done
• Done

Yes, it's a PAIN, but it works.

From what I understand the whole bookmarks stored in the server were completely broken, occ maintenance:repair wasn't removing them even though it said that it was. I'm guessing something changed, and well, it broke the database. Let's hope it doesn't happen again. I'm personally glad I didn't lost bookmarks.

[dsiminiuk]

I started having duplication issues in 4.4.5 where I would install the updated crx and then sync and suddenly I would have multiple copies downloaded of what was in the NC bookmarks. The NC bookmarks were fine, but the client was getting multiple copies.

The only way I found to clean it up was to delete all bookmarks from the browser, EXIT THE BROWSER (Vivaldi), then force sync from scratch. If I did not exit the browser after deleting all bookmarks and emptying the bookmark folder trash bin, the problem persisted. Something in memory perhaps?

[marcelklehr]

@tessus You ran into https://github.com/nextcloud/bookmarks/issues/1305 unfortunately :see_no_evil:

The repair step should now work as intended in v3.4.9.

Yes, it's a PAIN, but it works.

The repair step should do the same, just less painful (the irony, since it turned out to be more painful for a lot of people due to that bug.)

[tessus]

Luckily I had a db backup and a bookmarks backup, so in the end I did not lose any data.

I removed all oc_bookmarks* tables and removed the bookmarks app. Now I will install the latest bookmarks app and start syncing again.

the irony, since it turned out to be more painful for a lot of people due to that bug.

This statement cracked me up. It made my day. Thanks!

[tessus]

Btw, what's up with the floccus releases in the Chrome web store? The version is 4.3.0.

[tessus]

@marcelklehr When I try to install and enable the bookmarks app I get the following error:

An exception occurred while executing 'SELECT `p1`.`id` FROM `oc_bookmarks_shared_folders` `p1` LEFT JOIN `oc_bookmarks_shared_folders` `p2` ON (`p1`.`folder_id` = `p2`.`folder_id`) AND (`p1`.`user_id` = `p2`.`user_id`) WHERE `p2`.`id` < `p1`.`id`': SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.oc_bookmarks_shared_folders' doesn't exist

Yeah, of course, I removed all tables that started with oc_bookmarks*. Isn't the app supposed to create those tables?

[marcelklehr]

I removed all tables that started with oc_bookmarks*. Isn't the app supposed to create those tables?

Nextcloud apps track the database state in the oc_migrations table. As long as all migrations are still in that table, the app will not run any migrations.

[marcelklehr]

what's up with the floccus releases in the Chrome web store?

I wanted the dust around the v4.4.x series to settle until I upgrade another 10000 people to it...

[marcelklehr]

@wohali I feel really bad for you, especially, because I know you've been hitting bug after bug from the beginning. :disappointed:

[wohali]

Thanks @marcelklehr .

FYI I am still getting parentID errors under 4.4.6, with 3.4.9 on Nextcloud. However, now when I click on "Debug Logs," nothing happens, even if I wait for some minutes. Is there another way to access them? I went looking inside my profile directory and couldn't find them.

[wohali]

Still want to try and help here, and still in the same boat. Can't get debug logs out, and no changes since then.

[DerDanilo]

Same here. In Chrome and Edge the sync works fine but there the addon is only at 4.3.x. When will the update be pushed there?

[marcelklehr]

Sorry for the delay folks! I'm taking a bit of a break atm, but will resume work on floccus soon! I will not push v4.4.x to chrome/edge until the remaining issues have been resolved. I understand that the situation on firefox is somewhat undesirable at the moment. I'm sorry for the inconvenience and hope to resolve things soon! :heart:

[marcelklehr]

Hello Again!
I'm sorry for the delay on my part. I've now released v4.4.7 for firefox with a lot of new fixes, hopefully that will fix this as well. A few issues remain, but I'm confident I'll be able to tackle those soon. Thank you for your patience and your support :heart:

[marcelklehr]

v4.4.8 is out now, which is the result of more rigorous fuzzed testing, catching all sorts of bugs. I'm closing this for now. Feel free to shout or open a new issue if you continue to experience problems.

[wohali]

@marcelklehr Sorry to say that it still fails for me with "Failed to map parentId" errors on Windows.

2021-01-23T00:48:45.382Z Syncing failed with Failed to map parentId: aFgYyULtqjVs

I found the folder with that ID, renamed it, moved it to another folder, and tried to sync again. Same failure:

2021-01-23T00:54:51.916Z Syncing failed with Failed to map parentId: aFgYyULtqjVs

I can at least open the logs now and the sync doesn't hang - it just doesn't work.

Do you want a new ticket or to reopen this one?

[wohali]

I deleted the folder that was unhappy, removed the bookmarks out of it, and now the sync is hung again:

2021-01-23T01:00:10.881Z (local)ORDERFOLDER { id: 'lkbV-0vtGkQb',
  order: 
   [ { type: 'bookmark', id: 'osuQ4aF3VKT7' },
     { type: 'bookmark', id: '_V7FCWsz39n-' },
     { type: 'bookmark', id: 'HaOfDeWpYONX' },
     { type: 'bookmark', id: '0iWzLmKGyiS0' },
     { type: 'bookmark', id: 'mS6v1y2rq6nX' },
     { type: 'bookmark', id: 'xt6IY-43ge-G' },
     { type: 'bookmark', id: '3fiYdEdkUQwJ' },
     { type: 'bookmark', id: 'PPDiBZDppOOG' },
     { type: 'bookmark', id: '6E5_WZ2kQtD5' },
     { type: 'bookmark', id: 'xCeYDc7nnCTe' },
     { type: 'bookmark', id: 'aT_2x_M72dLg' },
     { type: 'folder', id: '8apvkV7xJLOR' } ] }
2021-01-23T01:00:11.618Z [Error: Nextcloud API error: 
{"status":"error","data":"invalid children order: To many children"}]
2021-01-23T01:00:11.619Z [Error: Nextcloud API error: 
{"status":"error","data":"invalid children order: To many children"}]
2021-01-23T01:00:11.619Z Syncing failed with Nextcloud API error: 
{"status":"error","data":"invalid children order: To many children"}
Nextcloud API error: 
{"status":"error","data":"invalid children order: To many children"}

In this state the Cancel Sync button did not work, and CPU use skyrocketed. I had to kill Firefox to get it to respond again.

I then forced a sync from scratch - which succeeded - and then did a second sync, only to get:

2021-01-23T01:11:11.665Z [Error: Incorrect argument types for bookmarks.move.]
2021-01-23T01:11:11.665Z Syncing failed with Incorrect argument types for bookmarks.move.

I'm disabling the plugin again. Please let me know if there's anything more I can do to help debug this.

[dsiminiuk]

@wohali
Try...
disable the extension.
delete all bookmarks
empty the bookmarks trash (if you have one in your browser)
close the browser
open the browser
set the extension to sync from scratch
click Sync

This works for me.

[wohali]

@dsiminiuk Thanks for the suggestion, but as I've told others on this and other issues, I'm not interested in doing that, because it means I will lose local bookmarks that haven't been synced up. I don't want to manually sync everything, there's thousands of bookmarks and it would take a lot of time.

[marcelklehr]

Wow, you're running into an awful lot of bugs :scream_cat:

I then forced a sync from scratch - which succeeded

This is one good sign at least. It means that your local and server bookmarks have been merged successfully, so you should now have the same bookmarks locally as on the server.

I'm trying to reproduce the rest of the bugs with my fuzzed tests, atm.

[marcelklehr]

Failed to map parentId: aFgYyULtqjVs

Fixed :heavy_check_mark:

CPU use skyrocketed

Fixed :heavy_check_mark:

Error: Incorrect argument types for bookmarks.move

Fixed :heavy_check_mark:

@wohali Can you try again with v4.4.9 when you have time?

This is one good sign at least. It means that your local and server bookmarks have been merged successfully, so you should now have the same bookmarks locally as on the server.

Can you confirm this?

[wohali]

4.4.9 started off working, with 2-3 syncs in a row 100% fine, but started failing again:

Debug logs is a 0 byte file now :(

[marcelklehr]

Then we have reduced this to a different issue: https://github.com/marcelklehr/floccus/issues/745

Let's continue over there.